-- Logs begin at Tue 2022-10-18 23:51:14 UTC, end at Wed 2022-10-19 00:57:35 UTC. -- Oct 19 00:31:38.540407 dv-cinder-master-3-iscsi systemd[1]: Starting Devstack devstack@n-api.service... Oct 19 00:31:38.554155 dv-cinder-master-3-iscsi devstack@n-api.service[110595]: [uWSGI] getting INI configuration from /etc/nova/nova-api-uwsgi.ini Oct 19 00:31:38.560079 dv-cinder-master-3-iscsi devstack@n-api.service[110595]: *** Starting uWSGI 2.0.18-debian (64bit) on [Wed Oct 19 00:31:38 2022] *** Oct 19 00:31:38.560079 dv-cinder-master-3-iscsi devstack@n-api.service[110595]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Oct 19 00:31:38.560079 dv-cinder-master-3-iscsi devstack@n-api.service[110595]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Oct 19 00:31:38.560079 dv-cinder-master-3-iscsi devstack@n-api.service[110595]: nodename: dv-cinder-master-3-iscsi Oct 19 00:31:38.560079 dv-cinder-master-3-iscsi devstack@n-api.service[110595]: machine: x86_64 Oct 19 00:31:38.560079 dv-cinder-master-3-iscsi devstack@n-api.service[110595]: clock source: unix Oct 19 00:31:38.560079 dv-cinder-master-3-iscsi devstack@n-api.service[110595]: pcre jit disabled Oct 19 00:31:38.560079 dv-cinder-master-3-iscsi devstack@n-api.service[110595]: detected number of CPU cores: 4 Oct 19 00:31:38.560079 dv-cinder-master-3-iscsi devstack@n-api.service[110595]: current working directory: / Oct 19 00:31:38.560079 dv-cinder-master-3-iscsi devstack@n-api.service[110595]: detected binary path: /usr/bin/uwsgi-core Oct 19 00:31:38.560079 dv-cinder-master-3-iscsi devstack@n-api.service[110595]: your processes number limit is 47680 Oct 19 00:31:38.560079 dv-cinder-master-3-iscsi devstack@n-api.service[110595]: your memory page size is 4096 bytes Oct 19 00:31:38.560079 dv-cinder-master-3-iscsi devstack@n-api.service[110595]: detected max file descriptor number: 2048 Oct 19 00:31:38.560079 dv-cinder-master-3-iscsi devstack@n-api.service[110595]: lock engine: pthread robust mutexes Oct 19 00:31:38.560079 dv-cinder-master-3-iscsi devstack@n-api.service[110595]: thunder lock: enabled Oct 19 00:31:38.560079 dv-cinder-master-3-iscsi devstack@n-api.service[110595]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/nova-api-wsgi.socket fd 4 Oct 19 00:31:38.560079 dv-cinder-master-3-iscsi devstack@n-api.service[110595]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Oct 19 00:31:38.604073 dv-cinder-master-3-iscsi devstack@n-api.service[110595]: Python main interpreter initialized at 0x561226abe9c0 Oct 19 00:31:38.604073 dv-cinder-master-3-iscsi devstack@n-api.service[110595]: python threads support enabled Oct 19 00:31:38.604073 dv-cinder-master-3-iscsi devstack@n-api.service[110595]: your server socket listen backlog is limited to 100 connections Oct 19 00:31:38.604073 dv-cinder-master-3-iscsi devstack@n-api.service[110595]: your mercy for graceful operations on workers is 90 seconds Oct 19 00:31:38.604641 dv-cinder-master-3-iscsi devstack@n-api.service[110595]: mapped 403077 bytes (393 KB) for 2 cores Oct 19 00:31:38.604641 dv-cinder-master-3-iscsi devstack@n-api.service[110595]: *** Operational MODE: preforking *** Oct 19 00:31:38.604888 dv-cinder-master-3-iscsi devstack@n-api.service[110595]: *** uWSGI is running in multiple interpreter mode *** Oct 19 00:31:38.604888 dv-cinder-master-3-iscsi devstack@n-api.service[110595]: spawned uWSGI master process (pid: 110595) Oct 19 00:31:38.605148 dv-cinder-master-3-iscsi systemd[1]: Started Devstack devstack@n-api.service. Oct 19 00:31:38.605729 dv-cinder-master-3-iscsi devstack@n-api.service[110595]: spawned uWSGI worker 1 (pid: 110596, cores: 1) Oct 19 00:31:38.606522 dv-cinder-master-3-iscsi devstack@n-api.service[110595]: spawned uWSGI worker 2 (pid: 110597, cores: 1) Oct 19 00:31:38.606522 dv-cinder-master-3-iscsi devstack@n-api.service[110595]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Oct 19 00:31:39.338275 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: Modules with known eventlet monkey patching issues were imported prior to eventlet monkey patching: urllib3. This warning can usually be ignored if the caller is only importing and not executing nova code. Oct 19 00:31:39.415730 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: Modules with known eventlet monkey patching issues were imported prior to eventlet monkey patching: urllib3. This warning can usually be ignored if the caller is only importing and not executing nova code. Oct 19 00:31:40.488250 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 19 00:31:40.488250 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: from cryptography.utils import int_from_bytes Oct 19 00:31:40.488863 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 19 00:31:40.488863 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: from cryptography.utils import int_from_bytes Oct 19 00:31:40.605382 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 19 00:31:40.605382 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: from cryptography.utils import int_from_bytes Oct 19 00:31:40.606160 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 19 00:31:40.606160 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: from cryptography.utils import int_from_bytes Oct 19 00:31:43.060172 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Oct 19 00:31:43.061047 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] Configuration options gathered from: {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Oct 19 00:31:43.061047 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] command line args: [] {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Oct 19 00:31:43.061376 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] config files: ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Oct 19 00:31:43.061640 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] ================================================================================ {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Oct 19 00:31:43.062042 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] allow_resize_to_same_host = True {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.062381 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] arq_binding_timeout = 300 {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.062737 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries = 60 {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.063065 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries_interval = 3 {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.063449 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] cert = self.pem {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.063770 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] compute_driver = None {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.064137 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] compute_monitors = [] {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.064502 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] config_dir = [] {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.064933 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] config_drive_format = iso9660 {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.065133 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] config_file = ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.065508 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] config_source = [] {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.065887 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] console_host = dv-cinder-master-3-iscsi {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.066217 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] control_exchange = nova {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.066611 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] cpu_allocation_ratio = None {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.066922 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] daemon = False {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.067294 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] debug = True {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.067607 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] default_access_ip_network_name = None {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.067932 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] default_availability_zone = nova {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.068287 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] default_ephemeral_format = None {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.068804 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'glanceclient=WARN', 'oslo.privsep.daemon=INFO'] {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.069254 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] default_schedule_zone = None {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.069626 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] disk_allocation_ratio = None {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.069959 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] enable_new_services = True {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.070375 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] enabled_apis = ['osapi_compute'] {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.070845 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] enabled_ssl_apis = [] {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.071110 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] flat_injected = False {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.071355 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] force_config_drive = True {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.071672 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] force_raw_images = True {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.072003 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] graceful_shutdown_timeout = 5 {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.072319 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] heal_instance_info_cache_interval = 60 {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.072741 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] host = dv-cinder-master-3-iscsi {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.073065 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] initial_cpu_allocation_ratio = 4.0 {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.073384 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] initial_disk_allocation_ratio = 1.0 {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.073696 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] initial_ram_allocation_ratio = 1.0 {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.074132 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] injected_network_template = /opt/stack/nova/nova/virt/interfaces.template {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.074443 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] instance_build_timeout = 0 {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.074804 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] instance_delete_interval = 300 {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.075130 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] instance_format = [instance: %(uuid)s] {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.075457 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] instance_name_template = instance-%08x {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.075791 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit = False {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.076101 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit_period = month {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.076417 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.076747 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] instances_path = /opt/stack/data/nova/instances {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.077077 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] internal_service_availability_zone = internal {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.077381 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] key = None {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.077693 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] live_migration_retry_count = 30 {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.078002 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] log_config_append = None {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.078327 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.078667 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] log_dir = None {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.078974 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] log_file = None {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.079227 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] log_options = True {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.079535 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval = 1 {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.079862 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval_type = days {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.080185 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] log_rotation_type = none {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.080510 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.080762 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.081015 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.081336 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.081659 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.081969 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] long_rpc_timeout = 1800 {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.082284 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_builds = 10 {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.082620 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_live_migrations = 1 {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.082926 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_snapshots = 5 {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.083235 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] max_local_block_devices = 3 {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.083540 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_count = 30 {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.084385 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_size_mb = 200 {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.084692 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] maximum_instance_delete_attempts = 5 {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.085019 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen = 0.0.0.0 {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.085338 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen_port = 8775 {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.085664 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] metadata_workers = 2 {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.085972 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] migrate_max_retries = -1 {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.086295 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] mkisofs_cmd = genisoimage {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.086734 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] my_block_storage_ip = 10.222.0.26 {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.086988 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] my_ip = 10.222.0.26 {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.087298 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] network_allocate_retries = 0 {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.087641 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] non_inheritable_image_properties = ['cache_in_nova', 'bittorrent'] {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.087963 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen = 0.0.0.0 {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.088275 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen_port = 8774 {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.088596 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_unique_server_name_scope = {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.088920 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_workers = 2 {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.089232 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] password_length = 12 {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.089541 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] periodic_enable = True {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.089849 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] periodic_fuzzy_delay = 60 {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.090170 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] pointer_model = usbtablet {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.090522 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] preallocate_images = none {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.090839 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] publish_errors = False {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.091091 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] pybasedir = /opt/stack/nova {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.091396 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] ram_allocation_ratio = None {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.091703 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_burst = 0 {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.092024 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_except_level = CRITICAL {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.092332 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_interval = 0 {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.092738 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] reboot_timeout = 0 {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.093133 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] reclaim_instance_interval = 0 {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.093593 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] record = None {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.093593 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] reimage_timeout_per_gb = 20 {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.094017 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] report_interval = 10 {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.094408 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] rescue_timeout = 0 {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.094408 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_cpus = 0 {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.094858 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_disk_mb = 0 {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.095082 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_memory_mb = 512 {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.095313 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] reserved_huge_pages = None {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.095619 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] resize_confirm_window = 0 {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.095923 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] resize_fs_using_block_device = False {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.096232 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] resume_guests_state_on_host_boot = False {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.096554 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] rootwrap_config = /etc/nova/rootwrap.conf {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.096873 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_action = reap {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.097437 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_poll_interval = 1800 {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.097745 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_timeout = 0 {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.098053 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] scheduler_instance_sync_interval = 120 {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.098358 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] service_down_time = 60 {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.098707 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] servicegroup_driver = db {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.099016 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] shelved_offload_time = 0 {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.099884 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] shelved_poll_interval = 3600 {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.100215 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] shutdown_timeout = 0 {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.100524 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] source_is_ipv6 = False {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.100833 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] ssl_only = False {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.101155 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] state_path = /opt/stack/data/nova {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.101465 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_interval = 600 {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.101772 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_pool_size = 1000 {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.102094 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] syslog_log_facility = LOG_USER {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.102400 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] tempdir = None {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.102733 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] timeout_nbd = 10 {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.103058 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] transport_url = **** {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.103367 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] update_resources_interval = 0 {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.103674 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] use_cow_images = True {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.103980 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] use_eventlog = False {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.104284 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] use_journal = False {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.104595 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] use_json = False {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.104902 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] use_rootwrap_daemon = False {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.105203 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] use_stderr = False {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.105513 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] use_syslog = False {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.105820 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] vcpu_pin_set = None {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.106121 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_is_fatal = True {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.106534 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_timeout = 300 {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.106835 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] virt_mkfs = [] {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.107140 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] volume_usage_poll_interval = 0 {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.107477 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] watch_log_file = False {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.107774 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] web = /usr/share/spice-html5 {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.108119 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.disable_process_locking = False {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.108456 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.lock_path = /opt/stack/data/nova {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.108798 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] api.auth_strategy = keystone {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.109119 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] api.compute_link_prefix = None {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.109451 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] api.config_drive_skip_versions = 1.0 2007-01-19 2007-03-01 2007-08-29 2007-10-10 2007-12-15 2008-02-01 2008-09-01 {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.109788 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] api.dhcp_domain = novalocal {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.110105 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] api.enable_instance_password = True {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.110427 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] api.glance_link_prefix = None {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.110762 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_fixed_size = 100 {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.111102 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_strategy = distributed {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.111408 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_per_project_cells = False {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.111719 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] api.list_records_by_skipping_down_cells = True {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.112032 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] api.local_metadata_per_cell = False {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.112350 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] api.max_limit = 1000 {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.112674 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] api.metadata_cache_expiration = 15 {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.113018 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] api.neutron_default_tenant_id = default {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.113331 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] api.use_forwarded_for = False {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.113654 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] api.use_neutron_default_nets = False {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.113969 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_connect_timeout = 5 {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.114333 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_failure_fatal = False {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.114770 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_read_timeout = 5 {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.115116 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_ssl_certfile = {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.115444 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_targets = [] {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.115768 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_jsonfile_path = None {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.116133 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_providers = ['StaticJSON'] {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.116477 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend = dogpile.cache.memcached {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.116799 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend_argument = **** {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.117122 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] cache.config_prefix = cache.oslo {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.117446 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] cache.dead_timeout = 60.0 {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.117765 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] cache.debug_cache_backend = False {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.118076 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_retry_client = False {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.118381 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_socket_keepalive = False {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.118752 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] cache.enabled = True {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.119072 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] cache.expiration_time = 600 {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.119383 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_attempts = 2 {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.119699 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_delay = 1.0 {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.120012 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_dead_retry = 300 {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.120335 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_password = {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.120649 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.120966 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.121319 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_maxsize = 10 {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.121683 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_unused_timeout = 60 {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.122001 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_sasl_enabled = False {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.122336 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_servers = ['localhost:11211'] {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.122683 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_socket_timeout = 1.0 {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.123017 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_username = {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.123378 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] cache.proxies = [] {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.123700 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_attempts = 2 {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.124016 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_delay = 0.0 {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.124396 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_count = 1 {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.124737 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_idle = 1 {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.125094 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_interval = 1 {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.125413 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_allowed_ciphers = None {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.125781 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_cafile = None {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.126088 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_certfile = None {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.126402 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_enabled = False {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.126740 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_keyfile = None {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.127072 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_section = None {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.127403 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_type = password {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.127707 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cafile = None {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.128043 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] cinder.catalog_info = volumev3::publicURL {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.128356 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] cinder.certfile = None {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.128668 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] cinder.collect_timing = False {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.128980 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cross_az_attach = True {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.129295 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] cinder.debug = False {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.129604 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] cinder.endpoint_template = None {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.129993 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] cinder.http_retries = 3 {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.130305 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] cinder.insecure = False {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.130662 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] cinder.keyfile = None {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.130994 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] cinder.os_region_name = RegionOne {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.131304 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] cinder.split_loggers = False {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.131612 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] cinder.timeout = None {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.131940 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] compute.consecutive_build_service_disable_threshold = 10 {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.132250 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_dedicated_set = None {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.132554 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_shared_set = None {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.132889 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] compute.image_type_exclude_list = [] {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.133184 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] compute.live_migration_wait_for_vif_plug = True {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.133498 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_concurrent_disk_ops = 0 {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.133808 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_disk_devices_to_attach = -1 {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.134120 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] compute.packing_host_numa_cells_allocation_strategy = False {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.134445 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] compute.provider_config_location = /etc/nova/provider_config/ {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.134789 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] compute.resource_provider_association_refresh = 300 {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.135095 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] compute.shutdown_retry_interval = 10 {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.135431 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] conductor.workers = 2 {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.135769 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] console.allowed_origins = [] {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.136072 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_ciphers = None {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.136400 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_minimum_version = default {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.136727 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] consoleauth.token_ttl = 600 {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.137045 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.cafile = None {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.137350 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.certfile = None {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.137657 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.collect_timing = False {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.137962 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retries = None {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.138265 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retry_delay = None {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.138595 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.endpoint_override = None {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.138906 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.insecure = False {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.139208 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.keyfile = None {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.139512 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.max_version = None {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.139823 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.min_version = None {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.140118 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.region_name = None {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.140422 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_name = None {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.140746 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_type = accelerator {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.141057 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.split_loggers = False {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.141358 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retries = None {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.141658 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retry_delay = None {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.141966 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.timeout = None {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.142310 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.valid_interfaces = ['internal', 'public'] {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.142654 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.version = None {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.142996 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] database.backend = sqlalchemy {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.143333 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] database.connection = **** {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.143657 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_debug = 0 {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.143981 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_parameters = {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.144297 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_recycle_time = 3600 {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.144621 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_trace = False {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.144940 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] database.db_inc_retry_interval = True {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.145244 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retries = 20 {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.145556 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retry_interval = 10 {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.145878 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] database.db_retry_interval = 1 {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.146801 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] database.max_overflow = 50 {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.147132 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] database.max_pool_size = 5 {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.147462 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] database.max_retries = 10 {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.147773 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_enable_ndb = False {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.148101 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_sql_mode = TRADITIONAL {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.148414 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] database.pool_timeout = None {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.148738 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] database.retry_interval = 10 {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.149045 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] database.slave_connection = **** {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.149362 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] database.sqlite_synchronous = True {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.149711 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] api_database.backend = sqlalchemy {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.150042 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection = **** {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.150363 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_debug = 0 {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.150715 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_parameters = {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.151027 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_recycle_time = 3600 {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.151346 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_trace = False {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.151662 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_inc_retry_interval = True {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.151974 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retries = 20 {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.152283 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retry_interval = 10 {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.152601 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_retry_interval = 1 {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.152914 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_overflow = 50 {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.153230 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_pool_size = 5 {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.153550 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_retries = 10 {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.153928 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_enable_ndb = False {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.154179 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_sql_mode = TRADITIONAL {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.154513 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] api_database.pool_timeout = None {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.154831 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] api_database.retry_interval = 10 {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.155146 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] api_database.slave_connection = **** {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.155463 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] api_database.sqlite_synchronous = True {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.155789 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] devices.enabled_mdev_types = [] {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.156129 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.cipher = aes-xts-plain64 {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.156440 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.enabled = False {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.156752 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.key_size = 512 {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.157072 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] glance.api_servers = None {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.157392 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] glance.cafile = None {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.157702 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] glance.certfile = None {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.158019 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] glance.collect_timing = False {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.158321 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retries = None {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.158660 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retry_delay = None {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.158971 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] glance.debug = False {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.159291 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] glance.default_trusted_certificate_ids = [] {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.159600 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_certificate_validation = False {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.159912 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_rbd_download = False {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.160212 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] glance.endpoint_override = None {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.160529 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] glance.insecure = False {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.160841 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] glance.keyfile = None {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.161147 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] glance.max_version = None {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.161447 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] glance.min_version = None {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.162555 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] glance.num_retries = 3 {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.162724 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_ceph_conf = {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.162955 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_connect_timeout = 5 {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.163285 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_pool = {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.163610 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_user = {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.163927 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] glance.region_name = None {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.164228 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_name = None {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.164550 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_type = image {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.164862 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] glance.split_loggers = False {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.165166 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retries = None {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.165473 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retry_delay = None {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.165779 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] glance.timeout = None {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.166125 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] glance.valid_interfaces = ['internal', 'public'] {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.166435 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] glance.verify_glance_signatures = False {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.166766 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] glance.version = None {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.167083 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] guestfs.debug = False {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.167410 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_cdrom = False {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.167719 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_inject_password = False {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.168040 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.dynamic_memory_ratio = 1.0 {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.168349 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_instance_metrics_collection = False {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.168660 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_remotefx = False {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.168982 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.instances_path_share = {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.169295 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.iscsi_initiator_list = [] {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.169605 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.limit_cpu_features = False {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.169919 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_count = 10 {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.170232 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_interval = 5 {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.170567 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_check_timeframe = 60 {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.170885 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_event_polling_interval = 2 {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.171202 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.qemu_img_cmd = qemu-img.exe {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.171511 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.use_multipath_io = False {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.171824 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_count = 10 {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.172137 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_interval = 5 {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.172440 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.vswitch_name = None {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.172755 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.wait_soft_reboot_seconds = 60 {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.173077 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] mks.enabled = False {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.173653 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] mks.mksproxy_base_url = http://127.0.0.1:6090/ {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.174004 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.manager_interval = 2400 {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.174303 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.precache_concurrency = 1 {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.174648 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_base_images = True {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.174993 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_original_minimum_age_seconds = 86400 {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.175314 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_resized_minimum_age_seconds = 3600 {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.175646 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.subdirectory_name = _base {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.175974 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_max_retries = 60 {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.176291 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_retry_interval = 2 {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.176596 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_section = None {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.176911 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_type = None {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.177220 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] ironic.cafile = None {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.177549 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] ironic.certfile = None {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.177863 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] ironic.collect_timing = False {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.178206 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retries = None {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.178541 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retry_delay = None {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.178853 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] ironic.endpoint_override = None {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.179166 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] ironic.insecure = False {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.179473 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] ironic.keyfile = None {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.179776 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] ironic.max_version = None {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.180079 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] ironic.min_version = None {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.180386 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] ironic.partition_key = None {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.180700 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] ironic.peer_list = [] {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.181006 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] ironic.region_name = None {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.181319 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] ironic.serial_console_state_timeout = 10 {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.181625 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_name = None {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.181945 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_type = baremetal {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.182258 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] ironic.split_loggers = False {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.182581 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retries = None {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.182890 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retry_delay = None {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.183202 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] ironic.timeout = None {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.183548 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] ironic.valid_interfaces = ['internal', 'public'] {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.183854 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] ironic.version = None {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.184194 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.backend = nova.keymgr.conf_key_mgr.ConfKeyManager {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.184523 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.fixed_key = **** {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.184876 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] barbican.auth_endpoint = http://localhost/identity/v3 {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.185175 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_api_version = None {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.185485 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint = None {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.185812 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint_type = public {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.186114 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_region_name = None {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.186420 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] barbican.cafile = None {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.186754 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] barbican.certfile = None {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.187068 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] barbican.collect_timing = False {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.187373 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] barbican.insecure = False {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.187673 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] barbican.keyfile = None {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.187983 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] barbican.number_of_retries = 60 {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.188293 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] barbican.retry_delay = 1 {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.188602 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] barbican.send_service_user_token = False {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.188914 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] barbican.split_loggers = False {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.189215 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] barbican.timeout = None {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.189533 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl = True {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.189837 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl_path = None {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.190155 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_section = None {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.190462 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_type = None {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.190795 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.cafile = None {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.191150 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.certfile = None {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.191515 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.collect_timing = False {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.191856 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.insecure = False {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.192169 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.keyfile = None {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.192568 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.split_loggers = False {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.192874 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.timeout = None {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.193242 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_role_id = None {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.193550 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_secret_id = None {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.193855 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] vault.cafile = None {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.194160 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] vault.certfile = None {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.194586 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] vault.collect_timing = False {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.194840 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] vault.insecure = False {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.195234 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] vault.keyfile = None {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.195463 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_mountpoint = secret {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.195889 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_version = 2 {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.196120 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] vault.namespace = None {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.196531 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] vault.root_token_id = None {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.196767 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] vault.split_loggers = False {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.196982 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] vault.ssl_ca_crt_file = None {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.197211 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] vault.timeout = None {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.197505 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] vault.use_ssl = False {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.197808 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] vault.vault_url = http://127.0.0.1:8200 {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.198221 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] keystone.cafile = None {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.198574 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] keystone.certfile = None {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.198890 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] keystone.collect_timing = False {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.199244 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retries = None {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.199596 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retry_delay = None {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.199924 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] keystone.endpoint_override = None {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.200174 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] keystone.insecure = False {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.200381 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] keystone.keyfile = None {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.200695 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] keystone.max_version = None {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.200986 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] keystone.min_version = None {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.201242 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] keystone.region_name = None {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.201552 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_name = None {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.201941 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_type = identity {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.202233 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] keystone.split_loggers = False {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.202527 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retries = None {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.202888 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retry_delay = None {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.203198 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] keystone.timeout = None {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.203589 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] keystone.valid_interfaces = ['internal', 'public'] {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.203852 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] keystone.version = None {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.204217 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.connection_uri = {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.204525 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_mode = None {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.204888 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_model_extra_flags = [] {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.205159 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_models = [] {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.205476 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_attempts = 8 {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.205846 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_timeout = 20 {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.206184 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_cachemodes = [] {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.206532 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_prefix = None {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.206831 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.enabled_perf_events = [] {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.207146 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.file_backed_memory = 0 {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.207544 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.gid_maps = [] {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.207856 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_disk_discard = None {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.208201 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_machine_type = None {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.208531 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_ceph_conf = {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.209481 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_poll_interval = 15 {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.209803 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_timeout = 600 {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.210159 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_store_name = {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.210589 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_pool = rbd {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.210942 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_type = default {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.211252 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_volume_group = None {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.211570 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_key = False {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.211929 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_partition = -2 {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.212287 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_password = False {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.212654 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iscsi_iface = None {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.213026 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iser_use_multipath = False {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.213378 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_bandwidth = 0 {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.213668 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_completion_timeout = 800 {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.213986 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime = 500 {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.214597 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_delay = 75 {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.214597 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_steps = 10 {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.215159 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_inbound_addr = None {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.215159 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_auto_converge = False {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.215721 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_post_copy = False {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.215939 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_scheme = None {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.216308 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_timeout_action = abort {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.216520 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_tunnelled = False {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.216823 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_uri = None {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.217139 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_with_native_tls = False {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.217561 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.max_queues = None {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.217960 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.mem_stats_period_seconds = 10 {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.218267 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_options = None {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.218674 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.219015 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_aoe_discover_tries = 3 {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.219416 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_iser_scan_tries = 5 {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.219787 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_memory_encrypted_guests = None {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.219998 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_nvme_discover_tries = 5 {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.220203 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_pcie_ports = 0 {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.220616 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_volume_scan_tries = 5 {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.220769 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.pmem_namespaces = [] {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.221036 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_client_cfg = None {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.221457 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_mount_point_base = /opt/stack/data/nova/mnt {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.221778 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_connect_timeout = 5 {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.222337 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retries = 12 {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.222687 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retry_interval = 5 {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.223084 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_secret_uuid = None {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.223284 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_user = None {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.223600 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.realtime_scheduler_priority = 1 {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.224000 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.remote_filesystem_transport = ssh {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.224863 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_image_id = None {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.225285 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_kernel_id = None {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.225598 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_ramdisk_id = None {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.225928 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rng_dev_path = /dev/urandom {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.226238 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rx_queue_size = None {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.226621 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_options = {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.227054 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.227382 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_compression = False {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.227679 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_image_format = None {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.228150 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshots_directory = /opt/stack/data/nova/instances/snapshots {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.228512 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sparse_logical_volumes = False {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.228900 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_enabled = False {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.229160 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_group = tss {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.229490 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_user = tss {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.229821 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sysinfo_serial = unique {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.230131 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.tx_queue_size = None {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.230448 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.uid_maps = [] {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.230869 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.use_virtio_for_bridges = True {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.231101 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.virt_type = kvm {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.231436 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear = zero {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.231878 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear_size = 0 {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.232049 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_use_multipath = False {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.232368 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_cache_path = None {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.232804 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_log_path = /var/log/vstorage/%(cluster_name)s/nova.log.gz {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.233101 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_group = qemu {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.233438 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_opts = [] {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.233670 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_perms = 0770 {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.234169 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_point_base = /opt/stack/data/nova/mnt {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.234417 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_user = stack {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.234806 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.wait_soft_reboot_seconds = 120 {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.235065 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_section = None {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.235422 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_type = password {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.235760 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] neutron.cafile = None {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.236067 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] neutron.certfile = None {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.236380 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] neutron.collect_timing = False {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.236693 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retries = None {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.237005 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retry_delay = None {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.237331 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] neutron.default_floating_pool = public {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.237749 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] neutron.endpoint_override = None {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.238056 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] neutron.extension_sync_interval = 600 {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.238374 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] neutron.http_retries = 3 {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.238712 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] neutron.insecure = False {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.239025 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] neutron.keyfile = None {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.239359 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] neutron.max_version = None {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.239702 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] neutron.metadata_proxy_shared_secret = **** {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.240110 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] neutron.min_version = None {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.240419 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] neutron.ovs_bridge = br-int {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.240798 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] neutron.physnets = [] {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.241101 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] neutron.region_name = RegionOne {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.241470 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_metadata_proxy = True {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.241786 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_name = None {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.242196 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_type = network {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.242525 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] neutron.split_loggers = False {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.242842 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retries = None {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.243233 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retry_delay = None {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.243512 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] neutron.timeout = None {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.243932 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] neutron.valid_interfaces = ['internal', 'public'] {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.244346 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] neutron.version = None {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.244569 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] notifications.bdms_in_notifications = False {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.244788 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] notifications.default_level = INFO {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.245045 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notification_format = unversioned {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.245360 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notify_on_state_change = None {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.245752 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] notifications.versioned_notifications_topics = ['versioned_notifications'] {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.246011 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] pci.alias = [] {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.246416 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] pci.device_spec = [] {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.246673 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] pci.report_in_placement = False {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.247077 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_section = None {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.247305 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_type = password {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.247576 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] placement.cafile = None {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.247869 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] placement.certfile = None {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.248194 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] placement.collect_timing = False {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.248494 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retries = None {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.248801 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retry_delay = None {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.249109 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] placement.endpoint_override = None {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.249418 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] placement.insecure = False {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.249741 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] placement.keyfile = None {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.250033 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] placement.max_version = None {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.250341 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] placement.min_version = None {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.250688 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] placement.region_name = RegionOne {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.250995 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_name = None {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.251321 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_type = placement {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.251634 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] placement.split_loggers = False {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.251941 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retries = None {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.252249 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retry_delay = None {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.252556 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] placement.timeout = None {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.252921 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] placement.valid_interfaces = ['internal', 'public'] {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.253277 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] placement.version = None {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.253665 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] quota.cores = 20 {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.253928 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] quota.count_usage_from_placement = False {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.254254 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] quota.driver = nova.quota.DbQuotaDriver {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.254599 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_content_bytes = 10240 {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.254921 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_path_length = 255 {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.255240 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_files = 5 {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.255855 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] quota.instances = 10 {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.256143 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] quota.key_pairs = 100 {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.256461 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] quota.metadata_items = 128 {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.256850 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] quota.ram = 51200 {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.257211 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] quota.recheck_quota = True {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.257531 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_group_members = 10 {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.257850 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_groups = 10 {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.258204 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] rdp.enabled = False {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.258857 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] rdp.html5_proxy_base_url = http://127.0.0.1:6083/ {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.259263 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.discover_hosts_in_cells_interval = -1 {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.259591 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.enable_isolated_aggregate_filtering = False {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.259908 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.image_metadata_prefilter = False {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.260282 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.limit_tenants_to_placement_aggregate = False {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.260561 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_attempts = 3 {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.260852 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_placement_results = 1000 {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.261210 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.placement_aggregate_required_for_tenants = False {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.261532 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_availability_zone = True {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.261893 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_image_type_support = False {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.262213 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_routed_network_aggregates = False {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.262572 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.workers = 2 {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.262894 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_namespace = None {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.263224 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_separator = . {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.263649 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.available_filters = ['nova.scheduler.filters.all_filters'] {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.263891 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.build_failure_weight_multiplier = 1000000.0 {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.264209 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cpu_weight_multiplier = 1.0 {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.264526 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cross_cell_move_weight_multiplier = 1000000.0 {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.264844 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.disk_weight_multiplier = 1.0 {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.265210 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.enabled_filters = ['AvailabilityZoneFilter', 'ComputeFilter', 'ComputeCapabilitiesFilter', 'ImagePropertiesFilter', 'ServerGroupAntiAffinityFilter', 'ServerGroupAffinityFilter', 'SameHostFilter', 'DifferentHostFilter'] {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.265572 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.host_subset_size = 1 {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.265895 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.image_properties_default_architecture = None {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.266217 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.io_ops_weight_multiplier = -1.0 {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.266598 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_hosts = [] {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.266958 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_images = [] {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.267280 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_instances_per_host = 50 {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.267678 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_io_ops_per_host = 8 {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.268006 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.pci_weight_multiplier = 1.0 {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.268325 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.ram_weight_multiplier = 1.0 {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.268642 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.restrict_isolated_hosts_to_isolated_images = True {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.268956 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.shuffle_best_same_weighed_hosts = False {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.269321 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_affinity_weight_multiplier = 1.0 {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.269666 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_anti_affinity_weight_multiplier = 1.0 {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.270002 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.track_instance_changes = False {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.270340 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.weight_classes = ['nova.scheduler.weights.all_weighers'] {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.270692 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] metrics.required = True {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.271022 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_multiplier = 1.0 {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.271894 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_of_unavailable = -10000.0 {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.272327 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_setting = [] {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.272892 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.base_url = ws://127.0.0.1:6083/ {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.273212 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.enabled = False {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.273550 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.port_range = 10000:20000 {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.273881 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.proxyclient_address = 127.0.0.1 {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.274208 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_host = 0.0.0.0 {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.274550 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_port = 6083 {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.274876 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_section = None {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.275212 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_type = password {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.275596 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] service_user.cafile = None {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.275912 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] service_user.certfile = None {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.276223 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] service_user.collect_timing = False {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.276534 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] service_user.insecure = False {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.276862 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] service_user.keyfile = None {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.277275 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] service_user.send_service_user_token = True {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.277556 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] service_user.split_loggers = False {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.277863 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] service_user.timeout = None {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.278184 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] spice.agent_enabled = True {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.278542 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] spice.enabled = False {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.279073 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_base_url = http://127.0.0.1:6082/spice_auto.html {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.279429 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_host = 0.0.0.0 {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.279753 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_port = 6082 {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.280085 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_listen = 127.0.0.1 {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.280416 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_proxyclient_address = 127.0.0.1 {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.280738 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.baseapi = None {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.281049 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.cert = None {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.281379 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.compute = auto {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.281690 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.conductor = None {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.282000 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.scheduler = None {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.282318 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_section = None {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.282655 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_type = None {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.282965 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.cafile = None {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.283273 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.certfile = None {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.283589 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.collect_timing = False {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.283902 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.insecure = False {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.284208 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.keyfile = None {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.284523 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.split_loggers = False {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.284830 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.timeout = None {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.285158 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] vmware.api_retry_count = 10 {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.285466 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] vmware.ca_file = None {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.285775 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cache_prefix = None {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.285873 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Oct 19 00:31:43.286292 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cluster_name = None {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.286366 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] Configuration options gathered from: {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Oct 19 00:31:43.286434 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] vmware.connection_pool_size = 10 {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.286704 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] vmware.console_delay_seconds = None {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.286852 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] command line args: [] {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Oct 19 00:31:43.287070 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] vmware.datastore_regex = None {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.287380 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_ip = None {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.287472 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] config files: ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Oct 19 00:31:43.287685 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_password = **** {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.287777 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] ================================================================================ {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Oct 19 00:31:43.288007 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_port = 443 {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.288242 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] allow_resize_to_same_host = True {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.288332 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_username = None {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.288616 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] vmware.insecure = False {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.288701 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] arq_binding_timeout = 300 {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.288920 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] vmware.integration_bridge = None {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.289013 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries = 60 {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.289233 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] vmware.maximum_objects = 100 {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.289414 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries_interval = 3 {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.289591 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_default_policy = None {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.289759 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] cert = self.pem {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.289923 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_enabled = False {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.290076 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] compute_driver = None {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.290241 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_wsdl_location = None {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.290398 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] compute_monitors = [] {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.290655 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_log_dir = /opt/vmware/vspc {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.290804 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] config_dir = [] {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.290981 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_proxy_uri = None {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.291212 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] config_drive_format = iso9660 {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.291294 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_service_uri = None {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.291480 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] config_file = ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.291602 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] vmware.task_poll_interval = 0.5 {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.291823 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] config_source = [] {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.291960 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] vmware.use_linked_clone = True {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.292152 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] console_host = dv-cinder-master-3-iscsi {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.292288 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_keymap = en-us {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.292475 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] control_exchange = nova {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.292685 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port = 5900 {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.292874 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] cpu_allocation_ratio = None {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.293003 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port_total = 10000 {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.293190 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] daemon = False {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.293357 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] vnc.auth_schemes = ['none'] {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.293509 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] debug = True {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.293677 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] vnc.enabled = True {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.293812 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] default_access_ip_network_name = None {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.294134 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] default_availability_zone = nova {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.294250 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_base_url = http://127.0.0.1:6080/vnc_auto.html {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.294441 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] default_ephemeral_format = None {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.294610 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_host = 0.0.0.0 {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.294937 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_port = 6080 {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.295025 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'glanceclient=WARN', 'oslo.privsep.daemon=INFO'] {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.295278 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_listen = 127.0.0.1 {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.295357 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] default_schedule_zone = None {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.295580 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] disk_allocation_ratio = None {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.295703 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_proxyclient_address = 127.0.0.1 {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.295888 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] enable_new_services = True {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.296006 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_ca_certs = None {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.296204 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_cert = None {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.296293 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] enabled_apis = ['osapi_compute'] {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.296512 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_key = None {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.296608 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] enabled_ssl_apis = [] {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.296816 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] flat_injected = False {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.296912 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_compute_service_check_for_ffu = False {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.297134 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] force_config_drive = True {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.297225 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_fallback_pcpu_query = False {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.297440 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] force_raw_images = True {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.297529 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_group_policy_check_upcall = False {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.297751 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_libvirt_livesnapshot = False {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.297839 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] graceful_shutdown_timeout = 5 {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.298054 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] heal_instance_info_cache_interval = 60 {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.298141 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_rootwrap = False {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.298364 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_numa_live_migration = False {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.298461 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] host = dv-cinder-master-3-iscsi {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.298707 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_qemu_monitor_announce_self = False {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.298801 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] initial_cpu_allocation_ratio = 4.0 {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.299020 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.ensure_libvirt_rbd_instance_dir_cleanup = False {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.299127 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] initial_disk_allocation_ratio = 1.0 {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.299335 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.handle_virt_lifecycle_events = True {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.299423 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] initial_ram_allocation_ratio = 1.0 {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.299647 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.libvirt_disable_apic = False {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.299775 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] injected_network_template = /opt/stack/nova/nova/virt/interfaces.template {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.299960 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.never_download_image_if_on_rbd = False {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.300085 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] instance_build_timeout = 0 {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.300274 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.reserve_disk_resource_for_image_cache = False {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.300391 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] instance_delete_interval = 300 {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.300586 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_cpu_compare_on_dest = False {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.300709 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] instance_format = [instance: %(uuid)s] {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.300896 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_hypervisor_version_check_on_lm = False {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.301034 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] instance_name_template = instance-%08x {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.301265 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.unified_limits_count_pcpu_as_vcpu = False {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.301369 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit = False {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.301520 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.wait_for_vif_plugged_event_during_hard_reboot = [] {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.301664 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit_period = month {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.301862 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.api_paste_config = /etc/nova/api-paste.ini {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.301986 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.302183 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.client_socket_timeout = 900 {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.302305 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] instances_path = /opt/stack/data/nova/instances {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.302671 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.default_pool_size = 1000 {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.302767 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] internal_service_availability_zone = internal {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.303005 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.keep_alive = True {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.303293 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] key = None {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.303382 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.max_header_line = 16384 {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.303624 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.secure_proxy_ssl_header = None {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.303714 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] live_migration_retry_count = 30 {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.303937 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_ca_file = None {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.304035 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] log_config_append = None {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.304249 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_cert_file = None {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.304348 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.304557 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] log_dir = None {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.304645 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_key_file = None {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.304864 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] log_file = None {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.304960 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.tcp_keepidle = 600 {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.305116 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] log_options = True {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.305212 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.wsgi_log_format = %(client_ip)s "%(request_line)s" status: %(status_code)s len: %(body_length)s time: %(wall_seconds).7f {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.305424 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval = 1 {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.305521 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] zvm.ca_file = None {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.305748 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval_type = days {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.305838 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] zvm.cloud_connector_url = None {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.306074 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] log_rotation_type = none {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.306398 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.306511 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] zvm.image_tmp_path = /opt/stack/data/nova/images {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.306679 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.306815 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] zvm.reachable_timeout = 300 {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.306930 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.307140 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.307250 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.307465 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.307562 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.307779 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.307869 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] long_rpc_timeout = 1800 {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.308107 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_process_name = {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.308200 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_builds = 10 {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.308437 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.308594 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_live_migrations = 1 {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.308738 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_snapshots = 5 {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.308827 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.309044 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] max_local_block_devices = 3 {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.309133 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_new_defaults = False {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.309348 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_count = 30 {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.309446 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_scope = False {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.309650 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_size_mb = 200 {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.309740 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_default_rule = default {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.309955 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] maximum_instance_delete_attempts = 5 {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.310044 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.310323 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen = 0.0.0.0 {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.310423 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_file = policy.yaml {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.310613 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen_port = 8775 {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.310724 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.310935 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] metadata_workers = 2 {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.311029 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.311240 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] migrate_max_retries = -1 {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.311478 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.311571 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.311654 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] mkisofs_cmd = genisoimage {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.311842 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.311933 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] my_block_storage_ip = 10.222.0.26 {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.312162 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_section = None {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.312259 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] my_ip = 10.222.0.26 {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.312475 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_type = None {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.312578 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] network_allocate_retries = 0 {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.312781 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.cafile = None {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.312871 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] non_inheritable_image_properties = ['cache_in_nova', 'bittorrent'] {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.313087 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.certfile = None {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.313185 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen = 0.0.0.0 {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.313399 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.collect_timing = False {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.313488 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen_port = 8774 {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.313705 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retries = None {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.313807 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_unique_server_name_scope = {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.314013 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retry_delay = None {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.314102 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_workers = 2 {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.314318 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] password_length = 12 {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.314406 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_id = None {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.314648 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_override = None {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.314751 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] periodic_enable = True {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.314954 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] periodic_fuzzy_delay = 60 {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.315044 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.insecure = False {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.315257 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.keyfile = None {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.315345 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] pointer_model = usbtablet {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.315562 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.max_version = None {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.315660 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] preallocate_images = none {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.315868 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.min_version = None {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.315957 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] publish_errors = False {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.316193 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] pybasedir = /opt/stack/nova {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.316294 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.region_name = None {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.316421 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] ram_allocation_ratio = None {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.316525 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_name = None {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.316730 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_burst = 0 {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.316835 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_type = None {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.317045 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_except_level = CRITICAL {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.317136 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.split_loggers = False {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.317348 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retries = None {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.317437 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_interval = 0 {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.317650 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] reboot_timeout = 0 {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.317738 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retry_delay = None {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.317953 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.timeout = None {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.318613 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] reclaim_instance_interval = 0 {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.318818 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.valid_interfaces = None {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.318964 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] record = None {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.319131 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.version = None {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.319362 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] reimage_timeout_per_gb = 20 {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.319502 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.319784 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] report_interval = 10 {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.319988 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] profiler.connection_string = messaging:// {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.320077 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] rescue_timeout = 0 {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.320224 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] profiler.enabled = False {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.320391 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_cpus = 0 {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.320555 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_doc_type = notification {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.320700 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_disk_mb = 0 {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.320869 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_size = 10000 {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.321010 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_memory_mb = 512 {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.321197 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_time = 2m {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.321317 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] reserved_huge_pages = None {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.321516 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] profiler.filter_error_trace = False {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.321641 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] resize_confirm_window = 0 {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.321841 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] profiler.hmac_keys = SECRET_KEY {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.321950 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] resize_fs_using_block_device = False {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.322165 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] profiler.sentinel_service_name = mymaster {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.322263 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] resume_guests_state_on_host_boot = False {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.322508 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] profiler.socket_timeout = 0.1 {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.322621 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] rootwrap_config = /etc/nova/rootwrap.conf {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.322825 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] profiler.trace_sqlalchemy = False {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.322914 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_action = reap {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.323146 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_poll_interval = 1800 {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.323244 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.323451 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_timeout = 0 {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.323550 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.323757 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] scheduler_instance_sync_interval = 120 {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.323846 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.324062 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] service_down_time = 60 {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.324156 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.324371 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.324466 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] servicegroup_driver = db {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.324673 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] shelved_offload_time = 0 {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.324765 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.324977 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] shelved_poll_interval = 3600 {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.325064 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_in_pthread = False {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.325302 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] shutdown_timeout = 0 {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.325423 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_rate = 3 {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.325608 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] source_is_ipv6 = False {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.325707 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_timeout_threshold = 600 {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.325913 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] ssl_only = False {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.326003 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_compression = None {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.326317 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] state_path = /opt/stack/data/nova {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.326430 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.326560 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_interval = 600 {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.326664 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.326869 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_pool_size = 1000 {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.326974 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.327185 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.327275 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] syslog_log_facility = LOG_USER {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.327484 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] tempdir = None {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.327573 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.327789 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] timeout_nbd = 10 {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.327878 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.328106 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] transport_url = **** {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.328196 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.328413 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] update_resources_interval = 0 {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.328505 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_delivery_limit = 0 {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.328716 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] use_cow_images = True {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.328814 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_queue = False {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.329020 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] use_eventlog = False {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.329111 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_bytes = 0 {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.329322 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] use_journal = False {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.329412 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_length = 0 {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.329626 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] use_json = False {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.329725 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.329930 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] use_rootwrap_daemon = False {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.330027 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.330234 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] use_stderr = False {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.330330 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.330562 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] use_syslog = False {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.330675 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.330864 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] vcpu_pin_set = None {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.330954 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl = False {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.331182 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_is_fatal = True {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.331271 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_ca_file = {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.331505 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_cert_file = {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.331669 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_timeout = 300 {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.331814 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] virt_mkfs = [] {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.331924 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_enforce_fips_mode = False {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.332120 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] volume_usage_poll_interval = 0 {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.332217 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_key_file = {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.332423 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] watch_log_file = False {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.332531 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_version = {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.332741 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] web = /usr/share/spice-html5 {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:31:43.332850 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.driver = ['messagingv2'] {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.333080 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.disable_process_locking = False {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.333186 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.retry = -1 {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.333410 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.lock_path = /opt/stack/data/nova {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.333509 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.topics = ['notifications'] {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.334318 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.transport_url = **** {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.334420 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] api.auth_strategy = keystone {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.334718 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler = None {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.334976 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] api.compute_link_prefix = None {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.335220 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler_interval = 1 {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.335324 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] api.config_drive_skip_versions = 1.0 2007-01-19 2007-03-01 2007-08-29 2007-10-10 2007-12-15 2008-02-01 2008-09-01 {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.335533 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.log_dir = None {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.335652 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] api.dhcp_domain = novalocal {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.335780 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=110596) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Oct 19 00:31:43.335977 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] api.enable_instance_password = True {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.336294 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] api.glance_link_prefix = None {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.336612 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_fixed_size = 100 {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.336937 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_strategy = distributed {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.337253 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_per_project_cells = False {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.337565 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] api.list_records_by_skipping_down_cells = True {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.337942 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] api.local_metadata_per_cell = False {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.338428 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] api.max_limit = 1000 {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.338918 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] api.metadata_cache_expiration = 15 {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.339245 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] api.neutron_default_tenant_id = default {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.339675 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] api.use_forwarded_for = False {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.340007 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] api.use_neutron_default_nets = False {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.340461 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_connect_timeout = 5 {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.341028 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_failure_fatal = False {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.341398 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_read_timeout = 5 {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.341706 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_ssl_certfile = {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.342151 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_targets = [] {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.342446 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_jsonfile_path = None {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.342850 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_providers = ['StaticJSON'] {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.343235 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend = dogpile.cache.memcached {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.344154 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend_argument = **** {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.344154 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] cache.config_prefix = cache.oslo {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.344154 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] cache.dead_timeout = 60.0 {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.344681 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] cache.debug_cache_backend = False {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.344894 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_retry_client = False {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.345039 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_socket_keepalive = False {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.345410 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] cache.enabled = True {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.345727 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] cache.expiration_time = 600 {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.346041 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_attempts = 2 {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.346443 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_delay = 1.0 {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.346698 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_dead_retry = 300 {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.347019 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_password = {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.347585 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.347897 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.348206 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_maxsize = 10 {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.348516 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_unused_timeout = 60 {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.348823 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_sasl_enabled = False {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.349163 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_servers = ['localhost:11211'] {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.349916 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_socket_timeout = 1.0 {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.350364 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_username = {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.351302 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] cache.proxies = [] {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.351625 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_attempts = 2 {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.351954 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_delay = 0.0 {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.352272 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_count = 1 {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.352586 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_idle = 1 {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.352897 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_interval = 1 {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.353209 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_allowed_ciphers = None {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.353501 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_cafile = None {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.353808 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_certfile = None {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.354117 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_enabled = False {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.354420 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_keyfile = None {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.354768 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_section = None {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.355097 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_type = password {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.355400 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cafile = None {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.355736 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] cinder.catalog_info = volumev3::publicURL {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.356039 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] cinder.certfile = None {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.356347 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] cinder.collect_timing = False {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.356680 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cross_az_attach = True {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.356962 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] cinder.debug = False {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.357266 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] cinder.endpoint_template = None {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.357580 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] cinder.http_retries = 3 {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.357888 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] cinder.insecure = False {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.358192 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] cinder.keyfile = None {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.358534 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] cinder.os_region_name = RegionOne {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.358849 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] cinder.split_loggers = False {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.359503 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] cinder.timeout = None {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.359503 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] compute.consecutive_build_service_disable_threshold = 10 {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.359870 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_dedicated_set = None {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.360068 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_shared_set = None {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.360383 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] compute.image_type_exclude_list = [] {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.360748 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] compute.live_migration_wait_for_vif_plug = True {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.360998 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_concurrent_disk_ops = 0 {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.361310 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_disk_devices_to_attach = -1 {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.361621 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] compute.packing_host_numa_cells_allocation_strategy = False {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.361949 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] compute.provider_config_location = /etc/nova/provider_config/ {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.362258 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] compute.resource_provider_association_refresh = 300 {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.362593 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] compute.shutdown_retry_interval = 10 {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.362932 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] conductor.workers = 2 {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.363266 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] console.allowed_origins = [] {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.363569 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_ciphers = None {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.363896 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_minimum_version = default {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.364220 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] consoleauth.token_ttl = 600 {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.364535 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.cafile = None {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.364839 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.certfile = None {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.365195 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.collect_timing = False {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.365729 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retries = None {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.366177 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retry_delay = None {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.366512 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.endpoint_override = None {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.366838 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.insecure = False {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.367140 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.keyfile = None {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.367437 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.max_version = None {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.367742 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.min_version = None {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.368047 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.region_name = None {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.368351 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_name = None {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.368682 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_type = accelerator {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.368995 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.split_loggers = False {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.369299 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retries = None {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.369603 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retry_delay = None {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.369907 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.timeout = None {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.370255 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.valid_interfaces = ['internal', 'public'] {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.370716 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.version = None {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.371068 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] database.backend = sqlalchemy {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.371406 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] database.connection = **** {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.371727 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_debug = 0 {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.372051 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_parameters = {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.372361 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_recycle_time = 3600 {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.372678 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_trace = False {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.372989 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] database.db_inc_retry_interval = True {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.373296 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retries = 20 {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.373607 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retry_interval = 10 {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.373917 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] database.db_retry_interval = 1 {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.374285 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] database.max_overflow = 50 {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.374645 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] database.max_pool_size = 5 {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.374999 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] database.max_retries = 10 {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.375319 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_enable_ndb = False {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.375649 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_sql_mode = TRADITIONAL {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.376122 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] database.pool_timeout = None {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.376520 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] database.retry_interval = 10 {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.376962 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] database.slave_connection = **** {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.377316 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] database.sqlite_synchronous = True {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.377671 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] api_database.backend = sqlalchemy {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.378011 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection = **** {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.378330 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_debug = 0 {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.378790 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_parameters = {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.379237 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_recycle_time = 3600 {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.379562 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_trace = False {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.379903 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_inc_retry_interval = True {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.380214 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retries = 20 {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.381074 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retry_interval = 10 {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.381464 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_retry_interval = 1 {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.381830 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_overflow = 50 {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.382348 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_pool_size = 5 {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.382706 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_retries = 10 {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.383049 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_enable_ndb = False {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.383382 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_sql_mode = TRADITIONAL {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.383699 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] api_database.pool_timeout = None {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.384021 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] api_database.retry_interval = 10 {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.384327 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] api_database.slave_connection = **** {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.384646 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] api_database.sqlite_synchronous = True {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.384980 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] devices.enabled_mdev_types = [] {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.385316 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.cipher = aes-xts-plain64 {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.385627 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.enabled = False {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.385944 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.key_size = 512 {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.386270 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] glance.api_servers = None {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.386613 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] glance.cafile = None {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.386962 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] glance.certfile = None {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.387367 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] glance.collect_timing = False {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.387608 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retries = None {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.387919 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retry_delay = None {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.388365 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] glance.debug = False {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.388684 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] glance.default_trusted_certificate_ids = [] {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.388998 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_certificate_validation = False {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.389314 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_rbd_download = False {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.389620 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] glance.endpoint_override = None {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.389936 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] glance.insecure = False {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.390246 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] glance.keyfile = None {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.390583 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] glance.max_version = None {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.390890 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] glance.min_version = None {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.391216 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] glance.num_retries = 3 {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.391540 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_ceph_conf = {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.391848 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_connect_timeout = 5 {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.392172 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_pool = {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.392653 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_user = {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.392955 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] glance.region_name = None {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.393530 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_name = None {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.393820 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_type = image {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.394344 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] glance.split_loggers = False {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.394738 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retries = None {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.395228 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retry_delay = None {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.395524 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] glance.timeout = None {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.395974 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] glance.valid_interfaces = ['internal', 'public'] {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.397131 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] glance.verify_glance_signatures = False {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.397832 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] glance.version = None {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.398222 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] guestfs.debug = False {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.398648 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_cdrom = False {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.398976 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_inject_password = False {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.399311 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.dynamic_memory_ratio = 1.0 {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.399623 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_instance_metrics_collection = False {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.399953 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_remotefx = False {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.400278 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.instances_path_share = {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.400599 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.iscsi_initiator_list = [] {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.400912 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.limit_cpu_features = False {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.401229 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_count = 10 {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.401543 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_interval = 5 {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.401856 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_check_timeframe = 60 {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.402169 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_event_polling_interval = 2 {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.402508 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.qemu_img_cmd = qemu-img.exe {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.402846 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.use_multipath_io = False {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.403193 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_count = 10 {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.403469 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_interval = 5 {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.403776 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.vswitch_name = None {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.404088 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.wait_soft_reboot_seconds = 60 {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.404408 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] mks.enabled = False {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.405002 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] mks.mksproxy_base_url = http://127.0.0.1:6090/ {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.405337 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.manager_interval = 2400 {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.405650 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.precache_concurrency = 1 {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.405969 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_base_images = True {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.406318 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_original_minimum_age_seconds = 86400 {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.406665 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_resized_minimum_age_seconds = 3600 {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.407002 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.subdirectory_name = _base {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.407347 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_max_retries = 60 {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.407661 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_retry_interval = 2 {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.407947 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_section = None {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.408254 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_type = None {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.408559 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] ironic.cafile = None {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.408862 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] ironic.certfile = None {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.409173 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] ironic.collect_timing = False {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.409488 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retries = None {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.409793 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retry_delay = None {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.410096 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] ironic.endpoint_override = None {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.410405 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] ironic.insecure = False {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.410775 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] ironic.keyfile = None {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.411081 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] ironic.max_version = None {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.411396 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] ironic.min_version = None {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.411689 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] ironic.partition_key = None {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.412044 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] ironic.peer_list = [] {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.412571 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] ironic.region_name = None {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.413050 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] ironic.serial_console_state_timeout = 10 {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.413365 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_name = None {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.413702 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_type = baremetal {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.414018 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] ironic.split_loggers = False {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.414325 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retries = None {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.414661 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retry_delay = None {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.414971 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] ironic.timeout = None {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.415320 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] ironic.valid_interfaces = ['internal', 'public'] {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.415630 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] ironic.version = None {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.415970 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.backend = nova.keymgr.conf_key_mgr.ConfKeyManager {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.416300 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.fixed_key = **** {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.416644 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] barbican.auth_endpoint = http://localhost/identity/v3 {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.416951 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_api_version = None {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.417256 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint = None {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.417582 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint_type = public {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.417888 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_region_name = None {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.418190 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] barbican.cafile = None {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.418525 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] barbican.certfile = None {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.418831 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] barbican.collect_timing = False {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.419139 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] barbican.insecure = False {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.419444 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] barbican.keyfile = None {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.419902 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] barbican.number_of_retries = 60 {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.420053 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] barbican.retry_delay = 1 {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.420362 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] barbican.send_service_user_token = False {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.420674 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] barbican.split_loggers = False {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.420972 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] barbican.timeout = None {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.421285 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl = True {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.421581 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl_path = None {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.421925 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_section = None {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.422205 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_type = None {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.422585 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.cafile = None {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.422866 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.certfile = None {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.423207 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.collect_timing = False {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.423490 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.insecure = False {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.423868 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.keyfile = None {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.424184 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.split_loggers = False {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.424483 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.timeout = None {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.424799 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_role_id = None {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.425103 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_secret_id = None {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.425401 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] vault.cafile = None {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.425705 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] vault.certfile = None {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.426012 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] vault.collect_timing = False {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.426326 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] vault.insecure = False {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.426647 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] vault.keyfile = None {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.427002 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_mountpoint = secret {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.427291 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_version = 2 {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.427651 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] vault.namespace = None {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.428160 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] vault.root_token_id = None {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.428647 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] vault.split_loggers = False {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.428958 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] vault.ssl_ca_crt_file = None {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.429349 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] vault.timeout = None {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.429666 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] vault.use_ssl = False {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.430001 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] vault.vault_url = http://127.0.0.1:8200 {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.430378 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] keystone.cafile = None {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.430725 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] keystone.certfile = None {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.431132 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] keystone.collect_timing = False {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.431449 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retries = None {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.431831 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retry_delay = None {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.432167 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] keystone.endpoint_override = None {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.432482 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] keystone.insecure = False {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.432860 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] keystone.keyfile = None {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.433167 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] keystone.max_version = None {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.433470 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] keystone.min_version = None {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.433834 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] keystone.region_name = None {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.434166 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_name = None {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.434556 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_type = identity {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.434847 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] keystone.split_loggers = False {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.435153 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retries = None {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.435460 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retry_delay = None {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.435770 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] keystone.timeout = None {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.436117 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] keystone.valid_interfaces = ['internal', 'public'] {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.436440 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] keystone.version = None {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.436790 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.connection_uri = {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.437103 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_mode = None {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.437388 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_model_extra_flags = [] {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.437738 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_models = [] {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.438030 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_attempts = 8 {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.438360 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_timeout = 20 {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.438703 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_cachemodes = [] {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.439003 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_prefix = None {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.439401 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.enabled_perf_events = [] {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.439828 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.file_backed_memory = 0 {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.440027 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.gid_maps = [] {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.440329 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_disk_discard = None {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.440668 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_machine_type = None {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.441053 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_ceph_conf = {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.441365 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_poll_interval = 15 {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.441676 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_timeout = 600 {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.442034 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_store_name = {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.442342 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_pool = rbd {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.442667 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_type = default {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.443760 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_volume_group = None {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.443931 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_key = False {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.444474 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_partition = -2 {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.444954 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_password = False {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.445357 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iscsi_iface = None {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.445658 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iser_use_multipath = False {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.446046 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_bandwidth = 0 {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.446712 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_completion_timeout = 800 {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.446851 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime = 500 {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.447196 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_delay = 75 {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.447457 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_steps = 10 {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.447840 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_inbound_addr = None {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.448154 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_auto_converge = False {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.448470 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_post_copy = False {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.448770 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_scheme = None {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.449097 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_timeout_action = abort {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.449405 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_tunnelled = False {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.449787 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_uri = None {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.450168 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_with_native_tls = False {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.450574 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.max_queues = None {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.450954 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.mem_stats_period_seconds = 10 {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.451260 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_options = None {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.451707 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.452083 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_aoe_discover_tries = 3 {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.452395 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_iser_scan_tries = 5 {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.452702 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_memory_encrypted_guests = None {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.453084 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_nvme_discover_tries = 5 {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.453397 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_pcie_ports = 0 {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.453719 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_volume_scan_tries = 5 {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.454059 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.pmem_namespaces = [] {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.454398 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_client_cfg = None {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.454977 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_mount_point_base = /opt/stack/data/nova/mnt {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.455329 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_connect_timeout = 5 {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.455736 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retries = 12 {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.456072 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retry_interval = 5 {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.456378 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_secret_uuid = None {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.456756 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_user = None {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.457063 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.realtime_scheduler_priority = 1 {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.457391 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.remote_filesystem_transport = ssh {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.457823 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_image_id = None {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.458171 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_kernel_id = None {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.458538 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_ramdisk_id = None {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.459771 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rng_dev_path = /dev/urandom {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.460196 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rx_queue_size = None {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.460545 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_options = {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.460957 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.461350 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_compression = False {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.461673 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_image_format = None {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.462077 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshots_directory = /opt/stack/data/nova/instances/snapshots {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.462472 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sparse_logical_volumes = False {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.462838 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_enabled = False {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.463238 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_group = tss {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.463684 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_user = tss {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.463975 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sysinfo_serial = unique {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.464390 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.tx_queue_size = None {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.464812 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.uid_maps = [] {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.465050 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.use_virtio_for_bridges = True {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.465371 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.virt_type = kvm {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.465770 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear = zero {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.466159 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear_size = 0 {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.466469 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_use_multipath = False {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.466858 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_cache_path = None {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.467205 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_log_path = /var/log/vstorage/%(cluster_name)s/nova.log.gz {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.467508 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_group = qemu {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.467888 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_opts = [] {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.468213 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_perms = 0770 {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.468702 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_point_base = /opt/stack/data/nova/mnt {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.469025 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_user = stack {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.469438 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.wait_soft_reboot_seconds = 120 {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.469728 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_section = None {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.470227 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_type = password {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.470522 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] neutron.cafile = None {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.470907 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] neutron.certfile = None {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.471219 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] neutron.collect_timing = False {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.471651 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retries = None {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.472028 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retry_delay = None {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.472609 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] neutron.default_floating_pool = public {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.472986 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] neutron.endpoint_override = None {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.473298 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] neutron.extension_sync_interval = 600 {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.473612 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] neutron.http_retries = 3 {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.473919 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] neutron.insecure = False {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.474224 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] neutron.keyfile = None {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.474679 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] neutron.max_version = None {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.475241 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] neutron.metadata_proxy_shared_secret = **** {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.475632 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] neutron.min_version = None {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.475963 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] neutron.ovs_bridge = br-int {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.476280 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] neutron.physnets = [] {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.476605 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] neutron.region_name = RegionOne {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.477047 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_metadata_proxy = True {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.477315 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_name = None {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.477711 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_type = network {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.478023 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] neutron.split_loggers = False {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.478402 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retries = None {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.478810 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retry_delay = None {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.479116 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] neutron.timeout = None {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.479463 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] neutron.valid_interfaces = ['internal', 'public'] {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.479770 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] neutron.version = None {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.480094 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] notifications.bdms_in_notifications = False {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.480488 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] notifications.default_level = INFO {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.480829 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notification_format = unversioned {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.481147 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notify_on_state_change = None {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.481551 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] notifications.versioned_notifications_topics = ['versioned_notifications'] {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.481948 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] pci.alias = [] {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.482344 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] pci.device_spec = [] {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.482688 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] pci.report_in_placement = False {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.483008 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_section = None {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.483332 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_type = password {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.483709 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] placement.cafile = None {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.484013 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] placement.certfile = None {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.484398 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] placement.collect_timing = False {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.484738 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retries = None {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.485106 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retry_delay = None {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.485416 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] placement.endpoint_override = None {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.485725 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] placement.insecure = False {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.486026 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] placement.keyfile = None {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.486327 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] placement.max_version = None {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.486668 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] placement.min_version = None {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.486997 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] placement.region_name = RegionOne {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.487301 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_name = None {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.487621 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_type = placement {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.487793 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: INFO dbcounter [None req-b8521d32-2727-489b-b296-9eba266c656f None None] Registered counter for database nova_api Oct 19 00:31:43.487923 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] placement.split_loggers = False {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.488218 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retries = None {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.488522 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retry_delay = None {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.488828 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] placement.timeout = None {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.489169 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] placement.valid_interfaces = ['internal', 'public'] {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.489541 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] placement.version = None {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.489904 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] quota.cores = 20 {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.490332 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] quota.count_usage_from_placement = False {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.490936 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] quota.driver = nova.quota.DbQuotaDriver {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.491263 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_content_bytes = 10240 {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.491718 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_path_length = 255 {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.492051 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_files = 5 {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.492439 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] quota.instances = 10 {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.492760 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] quota.key_pairs = 100 {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.493078 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] quota.metadata_items = 128 {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.493396 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] quota.ram = 51200 {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.493711 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] quota.recheck_quota = True {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.494030 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_group_members = 10 {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.494354 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_groups = 10 {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.494694 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] rdp.enabled = False {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.495264 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] rdp.html5_proxy_base_url = http://127.0.0.1:6083/ {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.495589 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.discover_hosts_in_cells_interval = -1 {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.495905 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.enable_isolated_aggregate_filtering = False {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.496219 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.image_metadata_prefilter = False {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.496531 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.limit_tenants_to_placement_aggregate = False {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.496845 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_attempts = 3 {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.497159 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_placement_results = 1000 {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.497472 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.placement_aggregate_required_for_tenants = False {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.497780 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_availability_zone = True {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.498091 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_image_type_support = False {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.498403 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_routed_network_aggregates = False {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.498758 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.workers = 2 {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.499089 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_namespace = None {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.499412 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_separator = . {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.499747 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.available_filters = ['nova.scheduler.filters.all_filters'] {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.500072 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.build_failure_weight_multiplier = 1000000.0 {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.500387 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cpu_weight_multiplier = 1.0 {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.500702 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cross_cell_move_weight_multiplier = 1000000.0 {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.501015 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.disk_weight_multiplier = 1.0 {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.501373 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.enabled_filters = ['AvailabilityZoneFilter', 'ComputeFilter', 'ComputeCapabilitiesFilter', 'ImagePropertiesFilter', 'ServerGroupAntiAffinityFilter', 'ServerGroupAffinityFilter', 'SameHostFilter', 'DifferentHostFilter'] {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.501686 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.host_subset_size = 1 {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.501998 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.image_properties_default_architecture = None {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.502304 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.io_ops_weight_multiplier = -1.0 {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.502648 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_hosts = [] {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.503291 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_images = [] {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.503614 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_instances_per_host = 50 {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.503923 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_io_ops_per_host = 8 {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.504245 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.pci_weight_multiplier = 1.0 {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.504617 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.ram_weight_multiplier = 1.0 {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.504852 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.restrict_isolated_hosts_to_isolated_images = True {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.505163 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.shuffle_best_same_weighed_hosts = False {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.505472 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_affinity_weight_multiplier = 1.0 {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.506863 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_anti_affinity_weight_multiplier = 1.0 {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.507268 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.track_instance_changes = False {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.507610 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.weight_classes = ['nova.scheduler.weights.all_weighers'] {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.507933 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] metrics.required = True {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.508254 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_multiplier = 1.0 {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.508567 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_of_unavailable = -10000.0 {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.508885 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_setting = [] {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.509451 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.base_url = ws://127.0.0.1:6083/ {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.509585 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG oslo_db.sqlalchemy.engines [None req-b8521d32-2727-489b-b296-9eba266c656f None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=110596) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 19 00:31:43.509778 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.enabled = False {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.510099 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.port_range = 10000:20000 {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.510425 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.proxyclient_address = 127.0.0.1 {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.510784 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_host = 0.0.0.0 {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.511099 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_port = 6083 {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.511418 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_section = None {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.511749 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_type = password {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.512049 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] service_user.cafile = None {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.512359 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] service_user.certfile = None {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.512669 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] service_user.collect_timing = False {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.512977 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] service_user.insecure = False {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.513281 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] service_user.keyfile = None {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.513602 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] service_user.send_service_user_token = True {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.513919 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] service_user.split_loggers = False {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.514280 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] service_user.timeout = None {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.514559 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] spice.agent_enabled = True {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.514938 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] spice.enabled = False {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.515488 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_base_url = http://127.0.0.1:6082/spice_auto.html {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.515964 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_host = 0.0.0.0 {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.516290 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_port = 6082 {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.516686 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_listen = 127.0.0.1 {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.516809 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG dbcounter [-] [110596] Writer thread running {{(pid=110596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 19 00:31:43.517065 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_proxyclient_address = 127.0.0.1 {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.517332 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.baseapi = None {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.517638 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.cert = None {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.517958 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.compute = auto {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.518265 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.conductor = None {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.518600 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.scheduler = None {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.518916 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_section = None {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.519231 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_type = None {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.519620 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.cafile = None {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.519835 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.certfile = None {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.520175 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.collect_timing = False {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.520607 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.insecure = False {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.520921 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.keyfile = None {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.521822 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.split_loggers = False {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.522342 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.timeout = None {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.522906 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] vmware.api_retry_count = 10 {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.523407 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] vmware.ca_file = None {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.523786 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cache_prefix = None {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.524101 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cluster_name = None {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.524410 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] vmware.connection_pool_size = 10 {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.524720 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] vmware.console_delay_seconds = None {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.525030 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] vmware.datastore_regex = None {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.525338 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_ip = None {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.525643 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_password = **** {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.525958 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_port = 443 {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.526264 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_username = None {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.526601 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] vmware.insecure = False {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.526911 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] vmware.integration_bridge = None {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.527221 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] vmware.maximum_objects = 100 {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.527575 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_default_policy = None {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.527890 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_enabled = False {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.528194 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_wsdl_location = None {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.528528 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_log_dir = /opt/vmware/vspc {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.528834 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_proxy_uri = None {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.529137 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_service_uri = None {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.529458 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] vmware.task_poll_interval = 0.5 {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.529770 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] vmware.use_linked_clone = True {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.530089 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_keymap = en-us {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.530402 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port = 5900 {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.530741 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port_total = 10000 {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.531089 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] vnc.auth_schemes = ['none'] {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.531405 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] vnc.enabled = True {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.531982 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_base_url = http://127.0.0.1:6080/vnc_auto.html {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.532351 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_host = 0.0.0.0 {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.532744 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_port = 6080 {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.533085 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_listen = 127.0.0.1 {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.533539 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_proxyclient_address = 127.0.0.1 {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.533926 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_ca_certs = None {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.534239 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_cert = None {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.534639 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_key = None {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.534976 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_compute_service_check_for_ffu = False {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.535360 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_fallback_pcpu_query = False {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.535748 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_group_policy_check_upcall = False {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.536063 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_libvirt_livesnapshot = False {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.536375 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_rootwrap = False {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.536687 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_numa_live_migration = False {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.537689 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_qemu_monitor_announce_self = False {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.538265 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.ensure_libvirt_rbd_instance_dir_cleanup = False {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.538720 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.handle_virt_lifecycle_events = True {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.539111 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.libvirt_disable_apic = False {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.539493 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.never_download_image_if_on_rbd = False {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.539932 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.reserve_disk_resource_for_image_cache = False {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.540325 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_cpu_compare_on_dest = False {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.540713 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_hypervisor_version_check_on_lm = False {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.541028 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.unified_limits_count_pcpu_as_vcpu = False {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.541431 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.wait_for_vif_plugged_event_during_hard_reboot = [] {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.541820 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.api_paste_config = /etc/nova/api-paste.ini {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.542145 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.client_socket_timeout = 900 {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.542589 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.default_pool_size = 1000 {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.543015 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.keep_alive = True {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.543338 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.max_header_line = 16384 {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.543650 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.secure_proxy_ssl_header = None {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.543965 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_ca_file = None {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.544275 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_cert_file = None {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.544586 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_key_file = None {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.544904 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.tcp_keepidle = 600 {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.545236 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.wsgi_log_format = %(client_ip)s "%(request_line)s" status: %(status_code)s len: %(body_length)s time: %(wall_seconds).7f {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.545549 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] zvm.ca_file = None {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.545908 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] zvm.cloud_connector_url = None {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.546597 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] zvm.image_tmp_path = /opt/stack/data/nova/images {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.546918 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] zvm.reachable_timeout = 300 {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.547242 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.547565 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.547882 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.548208 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_process_name = {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.549029 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.549029 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.549313 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_new_defaults = False {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.549455 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_scope = False {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.549762 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_default_rule = default {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.550099 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.550430 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_file = policy.yaml {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.550784 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.551091 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.551396 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.551700 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.552008 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.552325 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_section = None {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.553238 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_type = None {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.553788 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.cafile = None {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.554113 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.certfile = None {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.554429 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.collect_timing = False {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.554764 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retries = None {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.555072 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retry_delay = None {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.555378 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_id = None {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.555684 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_override = None {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.555994 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.insecure = False {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.556751 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.keyfile = None {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.556751 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.max_version = None {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.557054 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.min_version = None {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.557197 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.region_name = None {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.557459 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_name = None {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.557781 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_type = None {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.558074 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.split_loggers = False {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.558379 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retries = None {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.558710 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retry_delay = None {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.559017 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.timeout = None {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.559323 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.valid_interfaces = None {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.559624 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.version = None {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.559958 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.560292 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] profiler.connection_string = messaging:// {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.560656 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] profiler.enabled = False {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.560932 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_doc_type = notification {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.561241 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_size = 10000 {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.561563 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_time = 2m {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.561877 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] profiler.filter_error_trace = False {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.562206 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] profiler.hmac_keys = SECRET_KEY {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.562755 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] profiler.sentinel_service_name = mymaster {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.563085 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] profiler.socket_timeout = 0.1 {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.563397 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] profiler.trace_sqlalchemy = False {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.563734 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.564045 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.564443 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.564836 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.565225 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.565610 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.565923 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_in_pthread = False {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.566254 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_rate = 3 {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.566744 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_timeout_threshold = 600 {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.567059 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_compression = None {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.567461 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.567782 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.568133 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.568732 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.569199 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.569538 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.569869 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.570258 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_delivery_limit = 0 {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.570666 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_queue = False {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.571038 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_bytes = 0 {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.571370 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_length = 0 {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.571692 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.572030 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.572358 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.572772 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.573093 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl = False {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.573425 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_ca_file = {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.573754 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_cert_file = {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.574064 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_enforce_fips_mode = False {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.574467 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_key_file = {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.574906 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_version = {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.575333 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.driver = ['messagingv2'] {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.575649 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.retry = -1 {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.576065 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.topics = ['notifications'] {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.576463 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.transport_url = **** {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.576618 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.context [None req-b8521d32-2727-489b-b296-9eba266c656f None None] Found 2 cells: 00000000-0000-0000-0000-000000000000(cell0),3d98fd57-b040-4364-b7a0-5303a14589e8(cell1) {{(pid=110596) load_cells /opt/stack/nova/nova/context.py:464}} Oct 19 00:31:43.576850 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler = None {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.577241 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler_interval = 1 {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.577621 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.log_dir = None {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:31:43.577922 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=110597) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Oct 19 00:31:43.580416 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG oslo_concurrency.lockutils [None req-b8521d32-2727-489b-b296-9eba266c656f None None] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=110596) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:31:43.580891 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG oslo_concurrency.lockutils [None req-b8521d32-2727-489b-b296-9eba266c656f None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=110596) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:31:43.582469 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG oslo_concurrency.lockutils [None req-b8521d32-2727-489b-b296-9eba266c656f None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=110596) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:31:43.583357 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG oslo_concurrency.lockutils [None req-b8521d32-2727-489b-b296-9eba266c656f None None] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=110596) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:31:43.583886 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG oslo_concurrency.lockutils [None req-b8521d32-2727-489b-b296-9eba266c656f None None] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=110596) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:31:43.586013 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG oslo_concurrency.lockutils [None req-b8521d32-2727-489b-b296-9eba266c656f None None] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=110596) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:31:43.588163 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: INFO dbcounter [None req-b8521d32-2727-489b-b296-9eba266c656f None None] Registered counter for database nova_cell0 Oct 19 00:31:43.592494 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: INFO dbcounter [None req-b8521d32-2727-489b-b296-9eba266c656f None None] Registered counter for database nova_cell1 Oct 19 00:31:43.602810 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG oslo_db.sqlalchemy.engines [None req-b8521d32-2727-489b-b296-9eba266c656f None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=110596) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 19 00:31:43.603638 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG oslo_db.sqlalchemy.engines [None req-b8521d32-2727-489b-b296-9eba266c656f None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=110596) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 19 00:31:43.610325 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG dbcounter [-] [110596] Writer thread running {{(pid=110596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 19 00:31:43.737231 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: INFO dbcounter [None req-04e7fbe4-d196-4b1a-b09a-f0ea023f79b6 None None] Registered counter for database nova_api Oct 19 00:31:43.759971 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG oslo_db.sqlalchemy.engines [None req-04e7fbe4-d196-4b1a-b09a-f0ea023f79b6 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=110597) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 19 00:31:43.760908 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG dbcounter [-] [110596] Writer thread running {{(pid=110596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 19 00:31:43.769499 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG dbcounter [-] [110597] Writer thread running {{(pid=110597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 19 00:31:43.772145 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: INFO dbcounter [None req-b8521d32-2727-489b-b296-9eba266c656f None None] Registered counter for database nova_cell0 Oct 19 00:31:43.779180 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG oslo_db.sqlalchemy.engines [None req-b8521d32-2727-489b-b296-9eba266c656f None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=110596) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 19 00:31:43.784251 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG dbcounter [-] [110596] Writer thread running {{(pid=110596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 19 00:31:43.836358 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.context [None req-04e7fbe4-d196-4b1a-b09a-f0ea023f79b6 None None] Found 2 cells: 00000000-0000-0000-0000-000000000000(cell0),3d98fd57-b040-4364-b7a0-5303a14589e8(cell1) {{(pid=110597) load_cells /opt/stack/nova/nova/context.py:464}} Oct 19 00:31:43.840138 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG oslo_concurrency.lockutils [None req-04e7fbe4-d196-4b1a-b09a-f0ea023f79b6 None None] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=110597) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:31:43.840607 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG oslo_concurrency.lockutils [None req-04e7fbe4-d196-4b1a-b09a-f0ea023f79b6 None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=110597) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:31:43.842158 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG oslo_concurrency.lockutils [None req-04e7fbe4-d196-4b1a-b09a-f0ea023f79b6 None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=110597) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:31:43.842909 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG oslo_concurrency.lockutils [None req-04e7fbe4-d196-4b1a-b09a-f0ea023f79b6 None None] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=110597) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:31:43.843347 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG oslo_concurrency.lockutils [None req-04e7fbe4-d196-4b1a-b09a-f0ea023f79b6 None None] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=110597) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:31:43.845318 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG oslo_concurrency.lockutils [None req-04e7fbe4-d196-4b1a-b09a-f0ea023f79b6 None None] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=110597) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:31:43.847626 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: INFO dbcounter [None req-04e7fbe4-d196-4b1a-b09a-f0ea023f79b6 None None] Registered counter for database nova_cell0 Oct 19 00:31:43.852135 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: INFO dbcounter [None req-04e7fbe4-d196-4b1a-b09a-f0ea023f79b6 None None] Registered counter for database nova_cell1 Oct 19 00:31:43.859904 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG oslo_db.sqlalchemy.engines [None req-04e7fbe4-d196-4b1a-b09a-f0ea023f79b6 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=110597) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 19 00:31:43.865587 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG dbcounter [-] [110597] Writer thread running {{(pid=110597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 19 00:31:43.866469 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG oslo_db.sqlalchemy.engines [None req-04e7fbe4-d196-4b1a-b09a-f0ea023f79b6 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=110597) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 19 00:31:44.012135 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG dbcounter [-] [110597] Writer thread running {{(pid=110597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 19 00:31:44.021370 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: INFO dbcounter [None req-04e7fbe4-d196-4b1a-b09a-f0ea023f79b6 None None] Registered counter for database nova_cell0 Oct 19 00:31:44.034854 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG oslo_db.sqlalchemy.engines [None req-04e7fbe4-d196-4b1a-b09a-f0ea023f79b6 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=110597) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 19 00:31:44.042319 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG dbcounter [-] [110597] Writer thread running {{(pid=110597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 19 00:31:44.143632 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: WARNING keystonemiddleware.auth_token [None req-b8521d32-2727-489b-b296-9eba266c656f None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Oct 19 00:31:44.151116 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: WARNING keystonemiddleware.auth_token [None req-b8521d32-2727-489b-b296-9eba266c656f None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Oct 19 00:31:44.371934 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: WARNING keystonemiddleware.auth_token [None req-04e7fbe4-d196-4b1a-b09a-f0ea023f79b6 None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Oct 19 00:31:44.379930 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: WARNING keystonemiddleware.auth_token [None req-04e7fbe4-d196-4b1a-b09a-f0ea023f79b6 None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Oct 19 00:31:44.506364 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: WARNING keystonemiddleware.auth_token [None req-b8521d32-2727-489b-b296-9eba266c656f None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Oct 19 00:31:44.512681 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: WARNING keystonemiddleware.auth_token [None req-b8521d32-2727-489b-b296-9eba266c656f None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Oct 19 00:31:44.518156 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x561226abe9c0 pid: 110596 (default app) Oct 19 00:31:44.520340 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi [None req-b8521d32-2727-489b-b296-9eba266c656f None None] Calling method '>' {{(pid=110596) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:31:44.521824 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: INFO nova.api.openstack.requestlog [None req-b8521d32-2727-489b-b296-9eba266c656f None None] 10.222.0.26 "GET /compute/v2.1/" status: 200 len: 388 microversion: 2.1 time: 0.001839 Oct 19 00:31:44.522421 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: [pid: 110596|app: 0|req: 1/1] 10.222.0.26 () {50 vars in 788 bytes} [Wed Oct 19 00:31:44 2022] GET /compute/v2.1/ => generated 388 bytes in 4 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 19 00:31:44.727782 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: WARNING keystonemiddleware.auth_token [None req-04e7fbe4-d196-4b1a-b09a-f0ea023f79b6 None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Oct 19 00:31:44.733869 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: WARNING keystonemiddleware.auth_token [None req-04e7fbe4-d196-4b1a-b09a-f0ea023f79b6 None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Oct 19 00:31:44.739237 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x561226abe9c0 pid: 110597 (default app) Oct 19 00:33:11.189824 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi [None req-b8521d32-2727-489b-b296-9eba266c656f None None] Calling method '>' {{(pid=110596) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:33:11.191460 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: INFO nova.api.openstack.requestlog [None req-b8521d32-2727-489b-b296-9eba266c656f None None] 10.222.0.26 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.002009 Oct 19 00:33:11.191856 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: [pid: 110596|app: 0|req: 2/2] 10.222.0.26 () {54 vars in 931 bytes} [Wed Oct 19 00:33:11 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 19 00:33:11.205409 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG dbcounter [-] [110597] Writing DB stats nova_api:SELECT=3 {{(pid=110597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 00:33:11.207976 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG dbcounter [-] [110597] Writing DB stats nova_cell0:SELECT=3 {{(pid=110597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 00:33:11.208719 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG dbcounter [-] [110597] Writing DB stats nova_cell1:SELECT=3 {{(pid=110597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 00:33:11.209566 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG dbcounter [-] [110597] Writing DB stats nova_cell0:SELECT=3 {{(pid=110597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 00:33:12.512796 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi [None req-90013b40-2b86-4487-82b7-d590a3d7aa64 admin admin] Calling method '>' {{(pid=110597) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:33:12.552006 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: INFO nova.api.openstack.requestlog [None req-90013b40-2b86-4487-82b7-d590a3d7aa64 admin admin] 10.222.0.26 "GET /compute/v2.1/flavors/detail?is_public=True" status: 200 len: 15 microversion: 2.61 time: 1.352781 Oct 19 00:33:12.553012 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: [pid: 110597|app: 0|req: 1/3] 10.222.0.26 () {60 vars in 1292 bytes} [Wed Oct 19 00:33:11 2022] GET /compute/v2.1/flavors/detail?is_public=True => generated 15 bytes in 1355 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:33:15.772391 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi [None req-b8521d32-2727-489b-b296-9eba266c656f None None] Calling method '>' {{(pid=110596) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:33:15.775108 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: INFO nova.api.openstack.requestlog [None req-b8521d32-2727-489b-b296-9eba266c656f None None] 10.222.0.26 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.003233 Oct 19 00:33:15.777063 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: [pid: 110596|app: 0|req: 3/4] 10.222.0.26 () {54 vars in 931 bytes} [Wed Oct 19 00:33:15 2022] GET /compute/v2.1 => generated 388 bytes in 6 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 19 00:33:16.585557 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi [None req-a5893f8a-8304-4145-a6d4-287372d659b7 admin admin] Action: 'create', calling method: >, body: {"flavor": {"disk": 1, "OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0, "os-flavor-access:is_public": true, "ram": 256, "name": "cirros256", "id": "c1", "vcpus": 1, "swap": 0}} {{(pid=110597) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 19 00:33:16.654432 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: INFO nova.api.openstack.requestlog [None req-a5893f8a-8304-4145-a6d4-287372d659b7 admin admin] 10.222.0.26 "POST /compute/v2.1/flavors" status: 200 len: 407 microversion: 2.61 time: 0.870580 Oct 19 00:33:16.655880 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: [pid: 110597|app: 0|req: 2/5] 10.222.0.26 () {64 vars in 1283 bytes} [Wed Oct 19 00:33:15 2022] POST /compute/v2.1/flavors => generated 407 bytes in 873 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 19 00:33:16.672819 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG dbcounter [-] [110596] Writing DB stats nova_api:SELECT=3 {{(pid=110596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 00:33:16.673905 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG dbcounter [-] [110596] Writing DB stats nova_cell1:SELECT=3 {{(pid=110596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 00:33:16.677447 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG dbcounter [-] [110596] Writing DB stats nova_cell0:SELECT=3 {{(pid=110596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 00:33:16.678668 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG dbcounter [-] [110596] Writing DB stats nova_cell0:SELECT=6,nova_cell0:INSERT=1 {{(pid=110596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 00:33:16.930663 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi [None req-dd34c819-7271-40c1-b09f-9745bcdb558f admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=110596) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 19 00:33:17.005139 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: INFO nova.api.openstack.requestlog [None req-dd34c819-7271-40c1-b09f-9745bcdb558f admin admin] 10.222.0.26 "POST /compute/v2.1/flavors/c1/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.338642 Oct 19 00:33:17.005742 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: [pid: 110596|app: 0|req: 4/6] 10.222.0.26 () {64 vars in 1336 bytes} [Wed Oct 19 00:33:16 2022] POST /compute/v2.1/flavors/c1/os-extra_specs => generated 43 bytes in 340 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:33:19.804621 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi [None req-a5893f8a-8304-4145-a6d4-287372d659b7 admin admin] Calling method '>' {{(pid=110597) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:33:19.805925 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: INFO nova.api.openstack.requestlog [None req-a5893f8a-8304-4145-a6d4-287372d659b7 admin admin] 10.222.0.26 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001468 Oct 19 00:33:19.806815 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: [pid: 110597|app: 0|req: 3/7] 10.222.0.26 () {54 vars in 931 bytes} [Wed Oct 19 00:33:19 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 19 00:33:20.896276 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi [None req-365a8f73-82db-4e4a-9c4e-d04b694f2d88 admin admin] Action: 'create', calling method: >, body: {"flavor": {"os-flavor-access:is_public": true, "OS-FLV-EXT-DATA:ephemeral": 0, "id": "d1", "disk": 5, "ram": 512, "name": "ds512M", "vcpus": 1, "rxtx_factor": 1.0, "swap": 0}} {{(pid=110596) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 19 00:33:20.946783 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: INFO nova.api.openstack.requestlog [None req-365a8f73-82db-4e4a-9c4e-d04b694f2d88 admin admin] 10.222.0.26 "POST /compute/v2.1/flavors" status: 200 len: 404 microversion: 2.61 time: 1.120623 Oct 19 00:33:20.946783 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: [pid: 110596|app: 0|req: 5/8] 10.222.0.26 () {64 vars in 1283 bytes} [Wed Oct 19 00:33:19 2022] POST /compute/v2.1/flavors => generated 404 bytes in 1123 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 19 00:33:20.961196 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi [None req-5a155c55-dbcb-4990-87ac-6a5ae4dcbbf7 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=110597) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 19 00:33:21.007789 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: INFO nova.api.openstack.requestlog [None req-5a155c55-dbcb-4990-87ac-6a5ae4dcbbf7 admin admin] 10.222.0.26 "POST /compute/v2.1/flavors/d1/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.050333 Oct 19 00:33:21.008666 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: [pid: 110597|app: 0|req: 4/9] 10.222.0.26 () {64 vars in 1336 bytes} [Wed Oct 19 00:33:20 2022] POST /compute/v2.1/flavors/d1/os-extra_specs => generated 43 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:33:23.701968 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi [None req-365a8f73-82db-4e4a-9c4e-d04b694f2d88 admin admin] Calling method '>' {{(pid=110596) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:33:23.703010 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: INFO nova.api.openstack.requestlog [None req-365a8f73-82db-4e4a-9c4e-d04b694f2d88 admin admin] 10.222.0.26 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001488 Oct 19 00:33:23.703426 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: [pid: 110596|app: 0|req: 6/10] 10.222.0.26 () {54 vars in 931 bytes} [Wed Oct 19 00:33:23 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 19 00:33:24.224279 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi [None req-8dd44478-423b-49bf-9764-4cebab261fce admin admin] Action: 'create', calling method: >, body: {"flavor": {"id": "d2", "rxtx_factor": 1.0, "ram": 1024, "OS-FLV-EXT-DATA:ephemeral": 0, "vcpus": 1, "name": "ds1G", "swap": 0, "os-flavor-access:is_public": true, "disk": 10}} {{(pid=110597) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 19 00:33:24.245790 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: INFO nova.api.openstack.requestlog [None req-8dd44478-423b-49bf-9764-4cebab261fce admin admin] 10.222.0.26 "POST /compute/v2.1/flavors" status: 200 len: 404 microversion: 2.61 time: 0.535091 Oct 19 00:33:24.246345 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: [pid: 110597|app: 0|req: 5/11] 10.222.0.26 () {64 vars in 1283 bytes} [Wed Oct 19 00:33:23 2022] POST /compute/v2.1/flavors => generated 404 bytes in 536 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Oct 19 00:33:24.259926 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi [None req-b749b3af-9430-4e06-994e-1b674a73e553 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=110596) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 19 00:33:24.290995 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: INFO nova.api.openstack.requestlog [None req-b749b3af-9430-4e06-994e-1b674a73e553 admin admin] 10.222.0.26 "POST /compute/v2.1/flavors/d2/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.035262 Oct 19 00:33:24.291583 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: [pid: 110596|app: 0|req: 7/12] 10.222.0.26 () {64 vars in 1336 bytes} [Wed Oct 19 00:33:24 2022] POST /compute/v2.1/flavors/d2/os-extra_specs => generated 43 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:33:27.049759 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi [None req-8dd44478-423b-49bf-9764-4cebab261fce admin admin] Calling method '>' {{(pid=110597) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:33:27.052028 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: INFO nova.api.openstack.requestlog [None req-8dd44478-423b-49bf-9764-4cebab261fce admin admin] 10.222.0.26 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.002416 Oct 19 00:33:27.053603 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: [pid: 110597|app: 0|req: 6/13] 10.222.0.26 () {54 vars in 931 bytes} [Wed Oct 19 00:33:27 2022] GET /compute/v2.1 => generated 388 bytes in 5 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 19 00:33:27.619532 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi [None req-f6d674cc-e72d-49e0-858d-e88de5a5e89d admin admin] Action: 'create', calling method: >, body: {"flavor": {"swap": 0, "os-flavor-access:is_public": true, "name": "ds2G", "OS-FLV-EXT-DATA:ephemeral": 0, "disk": 10, "ram": 2048, "vcpus": 2, "id": "d3", "rxtx_factor": 1.0}} {{(pid=110596) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 19 00:33:27.643405 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: INFO nova.api.openstack.requestlog [None req-f6d674cc-e72d-49e0-858d-e88de5a5e89d admin admin] 10.222.0.26 "POST /compute/v2.1/flavors" status: 200 len: 404 microversion: 2.61 time: 0.579557 Oct 19 00:33:27.647015 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: [pid: 110596|app: 0|req: 8/14] 10.222.0.26 () {64 vars in 1283 bytes} [Wed Oct 19 00:33:27 2022] POST /compute/v2.1/flavors => generated 404 bytes in 585 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 19 00:33:27.657015 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi [None req-76fd0b97-7186-4df5-9d7b-a69afd99e12f admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=110597) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 19 00:33:27.693648 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: INFO nova.api.openstack.requestlog [None req-76fd0b97-7186-4df5-9d7b-a69afd99e12f admin admin] 10.222.0.26 "POST /compute/v2.1/flavors/d3/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.040399 Oct 19 00:33:27.694363 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: [pid: 110597|app: 0|req: 7/15] 10.222.0.26 () {64 vars in 1336 bytes} [Wed Oct 19 00:33:27 2022] POST /compute/v2.1/flavors/d3/os-extra_specs => generated 43 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 19 00:33:30.504145 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi [None req-f6d674cc-e72d-49e0-858d-e88de5a5e89d admin admin] Calling method '>' {{(pid=110596) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:33:30.505168 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: INFO nova.api.openstack.requestlog [None req-f6d674cc-e72d-49e0-858d-e88de5a5e89d admin admin] 10.222.0.26 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001471 Oct 19 00:33:30.505554 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: [pid: 110596|app: 0|req: 9/16] 10.222.0.26 () {54 vars in 931 bytes} [Wed Oct 19 00:33:30 2022] GET /compute/v2.1 => generated 388 bytes in 4 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 19 00:33:31.050716 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi [None req-24eb77ed-dc31-45db-b487-5c3f60d06efc admin admin] Action: 'create', calling method: >, body: {"flavor": {"name": "ds4G", "OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0, "os-flavor-access:is_public": true, "vcpus": 4, "swap": 0, "disk": 20, "ram": 4096, "id": "d4"}} {{(pid=110597) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 19 00:33:31.071991 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: INFO nova.api.openstack.requestlog [None req-24eb77ed-dc31-45db-b487-5c3f60d06efc admin admin] 10.222.0.26 "POST /compute/v2.1/flavors" status: 200 len: 404 microversion: 2.61 time: 0.559573 Oct 19 00:33:31.072550 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: [pid: 110597|app: 0|req: 8/17] 10.222.0.26 () {64 vars in 1283 bytes} [Wed Oct 19 00:33:30 2022] POST /compute/v2.1/flavors => generated 404 bytes in 561 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 19 00:33:31.085405 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi [None req-43df29ef-17a4-4bc3-bbdc-33c334e16e80 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=110596) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 19 00:33:31.119601 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: INFO nova.api.openstack.requestlog [None req-43df29ef-17a4-4bc3-bbdc-33c334e16e80 admin admin] 10.222.0.26 "POST /compute/v2.1/flavors/d4/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.037853 Oct 19 00:33:31.120155 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: [pid: 110596|app: 0|req: 10/18] 10.222.0.26 () {64 vars in 1336 bytes} [Wed Oct 19 00:33:31 2022] POST /compute/v2.1/flavors/d4/os-extra_specs => generated 43 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 19 00:33:34.030026 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi [None req-24eb77ed-dc31-45db-b487-5c3f60d06efc admin admin] Calling method '>' {{(pid=110597) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:33:34.031079 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: INFO nova.api.openstack.requestlog [None req-24eb77ed-dc31-45db-b487-5c3f60d06efc admin admin] 10.222.0.26 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001505 Oct 19 00:33:34.031496 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: [pid: 110597|app: 0|req: 9/19] 10.222.0.26 () {54 vars in 931 bytes} [Wed Oct 19 00:33:34 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 19 00:33:34.539423 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi [None req-31e5a704-1c44-41c9-bf4e-01b4eeebc3f3 admin admin] Calling method '>' {{(pid=110596) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:33:34.583317 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: INFO nova.api.openstack.requestlog [None req-31e5a704-1c44-41c9-bf4e-01b4eeebc3f3 admin admin] 10.222.0.26 "GET /compute/v2.1/flavors/detail?is_public=True" status: 200 len: 2106 microversion: 2.61 time: 0.543464 Oct 19 00:33:34.584137 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: [pid: 110596|app: 0|req: 11/20] 10.222.0.26 () {60 vars in 1292 bytes} [Wed Oct 19 00:33:34 2022] GET /compute/v2.1/flavors/detail?is_public=True => generated 2106 bytes in 545 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 19 00:33:37.285666 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi [None req-24eb77ed-dc31-45db-b487-5c3f60d06efc admin admin] Calling method '>' {{(pid=110597) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:33:37.286754 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: INFO nova.api.openstack.requestlog [None req-24eb77ed-dc31-45db-b487-5c3f60d06efc admin admin] 10.222.0.26 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001499 Oct 19 00:33:37.287179 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: [pid: 110597|app: 0|req: 10/21] 10.222.0.26 () {54 vars in 931 bytes} [Wed Oct 19 00:33:37 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 19 00:33:37.712587 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi [None req-51946fbc-d5e2-4ba5-9d02-94f28d359944 admin admin] Action: 'create', calling method: >, body: {"flavor": {"swap": 0, "OS-FLV-EXT-DATA:ephemeral": 0, "vcpus": 1, "os-flavor-access:is_public": true, "name": "m1.tiny", "rxtx_factor": 1.0, "id": "1", "disk": 1, "ram": 512}} {{(pid=110596) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 19 00:33:37.733718 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: INFO nova.api.openstack.requestlog [None req-51946fbc-d5e2-4ba5-9d02-94f28d359944 admin admin] 10.222.0.26 "POST /compute/v2.1/flavors" status: 200 len: 402 microversion: 2.61 time: 0.440137 Oct 19 00:33:37.734376 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: [pid: 110596|app: 0|req: 12/22] 10.222.0.26 () {64 vars in 1283 bytes} [Wed Oct 19 00:33:37 2022] POST /compute/v2.1/flavors => generated 402 bytes in 441 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 19 00:33:37.747095 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi [None req-951ca7d2-5b1c-4720-a53a-12dfcd4cee9f admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=110597) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 19 00:33:37.775738 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: INFO nova.api.openstack.requestlog [None req-951ca7d2-5b1c-4720-a53a-12dfcd4cee9f admin admin] 10.222.0.26 "POST /compute/v2.1/flavors/1/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.032375 Oct 19 00:33:37.776330 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: [pid: 110597|app: 0|req: 11/23] 10.222.0.26 () {64 vars in 1333 bytes} [Wed Oct 19 00:33:37 2022] POST /compute/v2.1/flavors/1/os-extra_specs => generated 43 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:33:40.483406 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi [None req-51946fbc-d5e2-4ba5-9d02-94f28d359944 admin admin] Calling method '>' {{(pid=110596) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:33:40.484411 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: INFO nova.api.openstack.requestlog [None req-51946fbc-d5e2-4ba5-9d02-94f28d359944 admin admin] 10.222.0.26 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001450 Oct 19 00:33:40.484825 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: [pid: 110596|app: 0|req: 13/24] 10.222.0.26 () {54 vars in 931 bytes} [Wed Oct 19 00:33:40 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 19 00:33:40.893692 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi [None req-79b52aed-5b5e-43e7-9b92-7cacfdaa63fd admin admin] Action: 'create', calling method: >, body: {"flavor": {"disk": 20, "swap": 0, "vcpus": 1, "ram": 2048, "rxtx_factor": 1.0, "os-flavor-access:is_public": true, "id": "2", "name": "m1.small", "OS-FLV-EXT-DATA:ephemeral": 0}} {{(pid=110597) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 19 00:33:40.915073 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: INFO nova.api.openstack.requestlog [None req-79b52aed-5b5e-43e7-9b92-7cacfdaa63fd admin admin] 10.222.0.26 "POST /compute/v2.1/flavors" status: 200 len: 405 microversion: 2.61 time: 0.422573 Oct 19 00:33:40.915618 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: [pid: 110597|app: 0|req: 12/25] 10.222.0.26 () {64 vars in 1283 bytes} [Wed Oct 19 00:33:40 2022] POST /compute/v2.1/flavors => generated 405 bytes in 424 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Oct 19 00:33:40.928256 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi [None req-1497fd55-fde1-4d84-b046-e1dbc2a6385b admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=110596) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 19 00:33:40.962260 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: INFO nova.api.openstack.requestlog [None req-1497fd55-fde1-4d84-b046-e1dbc2a6385b admin admin] 10.222.0.26 "POST /compute/v2.1/flavors/2/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.037804 Oct 19 00:33:40.963049 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: [pid: 110596|app: 0|req: 14/26] 10.222.0.26 () {64 vars in 1333 bytes} [Wed Oct 19 00:33:40 2022] POST /compute/v2.1/flavors/2/os-extra_specs => generated 43 bytes in 40 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:33:43.757967 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi [None req-79b52aed-5b5e-43e7-9b92-7cacfdaa63fd admin admin] Calling method '>' {{(pid=110597) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:33:43.759024 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: INFO nova.api.openstack.requestlog [None req-79b52aed-5b5e-43e7-9b92-7cacfdaa63fd admin admin] 10.222.0.26 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001593 Oct 19 00:33:43.762561 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: [pid: 110597|app: 0|req: 13/27] 10.222.0.26 () {54 vars in 931 bytes} [Wed Oct 19 00:33:43 2022] GET /compute/v2.1 => generated 388 bytes in 5 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 19 00:33:44.257362 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi [None req-639dd38a-ec9d-46c1-8bcd-44374d8cbeb2 admin admin] Action: 'create', calling method: >, body: {"flavor": {"disk": 40, "vcpus": 2, "id": "3", "ram": 4096, "rxtx_factor": 1.0, "swap": 0, "os-flavor-access:is_public": true, "OS-FLV-EXT-DATA:ephemeral": 0, "name": "m1.medium"}} {{(pid=110596) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 19 00:33:44.314868 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: INFO nova.api.openstack.requestlog [None req-639dd38a-ec9d-46c1-8bcd-44374d8cbeb2 admin admin] 10.222.0.26 "POST /compute/v2.1/flavors" status: 200 len: 406 microversion: 2.61 time: 0.544328 Oct 19 00:33:44.318162 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: [pid: 110596|app: 0|req: 15/28] 10.222.0.26 () {64 vars in 1283 bytes} [Wed Oct 19 00:33:43 2022] POST /compute/v2.1/flavors => generated 406 bytes in 551 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 19 00:33:44.354546 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi [None req-532fc672-9e05-4360-9020-f3b1fe1b099e admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=110597) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 19 00:33:44.360839 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG dbcounter [-] [110597] Writing DB stats nova_api:SELECT=33,nova_api:INSERT=7,nova_api:SAVEPOINT=3,nova_api:RELEASE=3 {{(pid=110597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 00:33:44.389434 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: INFO nova.api.openstack.requestlog [None req-532fc672-9e05-4360-9020-f3b1fe1b099e admin admin] 10.222.0.26 "POST /compute/v2.1/flavors/3/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.039161 Oct 19 00:33:44.390014 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: [pid: 110597|app: 0|req: 14/29] 10.222.0.26 () {64 vars in 1333 bytes} [Wed Oct 19 00:33:44 2022] POST /compute/v2.1/flavors/3/os-extra_specs => generated 43 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:33:47.044351 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi [None req-639dd38a-ec9d-46c1-8bcd-44374d8cbeb2 admin admin] Calling method '>' {{(pid=110596) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:33:47.045476 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: INFO nova.api.openstack.requestlog [None req-639dd38a-ec9d-46c1-8bcd-44374d8cbeb2 admin admin] 10.222.0.26 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001561 Oct 19 00:33:47.046309 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: [pid: 110596|app: 0|req: 16/30] 10.222.0.26 () {54 vars in 931 bytes} [Wed Oct 19 00:33:47 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 19 00:33:47.502629 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi [None req-6fab5133-c1ff-4f9f-be02-0af494b68e07 admin admin] Action: 'create', calling method: >, body: {"flavor": {"vcpus": 4, "name": "m1.large", "os-flavor-access:is_public": true, "id": "4", "ram": 8192, "rxtx_factor": 1.0, "OS-FLV-EXT-DATA:ephemeral": 0, "swap": 0, "disk": 80}} {{(pid=110597) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 19 00:33:47.522225 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: INFO nova.api.openstack.requestlog [None req-6fab5133-c1ff-4f9f-be02-0af494b68e07 admin admin] 10.222.0.26 "POST /compute/v2.1/flavors" status: 200 len: 405 microversion: 2.61 time: 0.469870 Oct 19 00:33:47.522809 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: [pid: 110597|app: 0|req: 15/31] 10.222.0.26 () {64 vars in 1283 bytes} [Wed Oct 19 00:33:47 2022] POST /compute/v2.1/flavors => generated 405 bytes in 471 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 19 00:33:47.535259 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi [None req-6661378a-16c7-4c35-a5f8-1040007de350 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=110596) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 19 00:33:47.545544 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG dbcounter [-] [110596] Writing DB stats nova_api:SELECT=39,nova_api:SAVEPOINT=4,nova_api:INSERT=8,nova_api:RELEASE=4 {{(pid=110596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 00:33:47.563017 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: INFO nova.api.openstack.requestlog [None req-6661378a-16c7-4c35-a5f8-1040007de350 admin admin] 10.222.0.26 "POST /compute/v2.1/flavors/4/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.031837 Oct 19 00:33:47.563598 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: [pid: 110596|app: 0|req: 17/32] 10.222.0.26 () {64 vars in 1333 bytes} [Wed Oct 19 00:33:47 2022] POST /compute/v2.1/flavors/4/os-extra_specs => generated 43 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:33:50.237813 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi [None req-6fab5133-c1ff-4f9f-be02-0af494b68e07 admin admin] Calling method '>' {{(pid=110597) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:33:50.238920 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: INFO nova.api.openstack.requestlog [None req-6fab5133-c1ff-4f9f-be02-0af494b68e07 admin admin] 10.222.0.26 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001603 Oct 19 00:33:50.239606 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: [pid: 110597|app: 0|req: 16/33] 10.222.0.26 () {54 vars in 931 bytes} [Wed Oct 19 00:33:50 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 19 00:33:50.731125 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi [None req-35678c71-5e28-41d0-8c0f-884a9db3dbc9 admin admin] Action: 'create', calling method: >, body: {"flavor": {"ram": 16384, "disk": 160, "rxtx_factor": 1.0, "name": "m1.xlarge", "OS-FLV-EXT-DATA:ephemeral": 0, "id": "5", "swap": 0, "os-flavor-access:is_public": true, "vcpus": 8}} {{(pid=110596) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 19 00:33:50.751759 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: INFO nova.api.openstack.requestlog [None req-35678c71-5e28-41d0-8c0f-884a9db3dbc9 admin admin] 10.222.0.26 "POST /compute/v2.1/flavors" status: 200 len: 408 microversion: 2.61 time: 0.504687 Oct 19 00:33:50.752386 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: [pid: 110596|app: 0|req: 18/34] 10.222.0.26 () {64 vars in 1283 bytes} [Wed Oct 19 00:33:50 2022] POST /compute/v2.1/flavors => generated 408 bytes in 507 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 19 00:33:50.764675 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi [None req-3843a1d2-7ba2-41e7-8908-994f324139f4 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=110597) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 19 00:33:50.796660 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: INFO nova.api.openstack.requestlog [None req-3843a1d2-7ba2-41e7-8908-994f324139f4 admin admin] 10.222.0.26 "POST /compute/v2.1/flavors/5/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.035684 Oct 19 00:33:50.797861 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: [pid: 110597|app: 0|req: 17/35] 10.222.0.26 () {64 vars in 1333 bytes} [Wed Oct 19 00:33:50 2022] POST /compute/v2.1/flavors/5/os-extra_specs => generated 43 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:34:07.625164 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi [None req-35678c71-5e28-41d0-8c0f-884a9db3dbc9 admin admin] Calling method '>' {{(pid=110596) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:34:07.626792 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: INFO nova.api.openstack.requestlog [None req-35678c71-5e28-41d0-8c0f-884a9db3dbc9 admin admin] 10.222.0.26 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.002104 Oct 19 00:34:07.627199 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: [pid: 110596|app: 0|req: 19/36] 10.222.0.26 () {54 vars in 931 bytes} [Wed Oct 19 00:34:07 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 19 00:34:07.638412 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG dbcounter [-] [110597] Writing DB stats nova_api:SELECT=14,nova_api:SAVEPOINT=2,nova_api:INSERT=3,nova_api:RELEASE=2 {{(pid=110597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 00:34:08.047938 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi [None req-f757d910-261f-4d37-8960-baaac1c39116 admin admin] Calling method '>' {{(pid=110597) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:34:08.071174 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: INFO nova.api.openstack.requestlog [None req-f757d910-261f-4d37-8960-baaac1c39116 admin admin] 10.222.0.26 "GET /compute/v2.1/flavors/detail?is_public=True" status: 200 len: 4202 microversion: 2.61 time: 0.437242 Oct 19 00:34:08.071720 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: [pid: 110597|app: 0|req: 18/37] 10.222.0.26 () {60 vars in 1292 bytes} [Wed Oct 19 00:34:07 2022] GET /compute/v2.1/flavors/detail?is_public=True => generated 4202 bytes in 438 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 19 00:34:09.432281 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi [None req-35678c71-5e28-41d0-8c0f-884a9db3dbc9 admin admin] Calling method '>' {{(pid=110596) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:34:09.433276 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: INFO nova.api.openstack.requestlog [None req-35678c71-5e28-41d0-8c0f-884a9db3dbc9 admin admin] 10.222.0.26 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001463 Oct 19 00:34:09.433671 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: [pid: 110596|app: 0|req: 20/38] 10.222.0.26 () {56 vars in 1074 bytes} [Wed Oct 19 00:34:09 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 19 00:34:10.045703 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi [None req-48d9d515-0a0d-45d5-acd9-8763116ece73 admin admin] Calling method '>' {{(pid=110597) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:34:10.052429 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: INFO nova.api.openstack.wsgi [None req-48d9d515-0a0d-45d5-acd9-8763116ece73 admin admin] HTTP exception thrown: Flavor 42 could not be found. Oct 19 00:34:10.055769 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi [None req-48d9d515-0a0d-45d5-acd9-8763116ece73 admin admin] Request method failure captured: Oct 19 00:34:10.055769 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: request: GET /compute/v2.1/flavors/42 HTTP/1.1 Oct 19 00:34:10.055769 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: Accept: application/json Oct 19 00:34:10.055769 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: Accept-Encoding: gzip, deflate Oct 19 00:34:10.055769 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: Connection: keep-alive Oct 19 00:34:10.055769 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: Content-Length: 0 Oct 19 00:34:10.055769 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: Host: 10.222.0.26 Oct 19 00:34:10.055769 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: Openstack-Api-Version: compute 2.92 Oct 19 00:34:10.055769 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: Openstack-System-Scope: None Oct 19 00:34:10.055769 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: User-Agent: python-novaclient Oct 19 00:34:10.055769 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: X-Auth-Token: gAAAAABjT0YBbF7A-FWAsBy9OGnjZ1x2k_Xa4BOTENwY0c4KaGseYq8uKMG3_lWZTmtPbD6-mm3ivr7Z_KPXcIJHnozSls5OnmMjlcYNNHT2JmOBUAA3ej91OAId4mFldYMJj2YEUnHK31FipQtTj-5O9zzq8FZk-F8i-OS5X1Yp3V_GdaEhlls Oct 19 00:34:10.055769 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: X-Domain-Id: None Oct 19 00:34:10.055769 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: X-Domain-Name: None Oct 19 00:34:10.055769 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: X-Identity-Status: Confirmed Oct 19 00:34:10.055769 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: X-Is-Admin-Project: True Oct 19 00:34:10.055769 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: X-Openstack-Nova-Api-Version: 2.92 Oct 19 00:34:10.055769 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: X-Project-Domain-Id: default Oct 19 00:34:10.055769 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: X-Project-Domain-Name: Default Oct 19 00:34:10.055769 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: X-Project-Id: d417ad1948a04e309383739147354388 Oct 19 00:34:10.055769 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: X-Project-Name: admin Oct 19 00:34:10.055769 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: X-Role: admin,member,reader Oct 19 00:34:10.055769 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: X-Roles: admin,member,reader Oct 19 00:34:10.057409 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/d417ad1948a04e309383739147354388"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/d417ad1948a04e309383739147354388"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/d417ad1948a04e309383739147354388"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_d417ad1948a04e309383739147354388"}]}] Oct 19 00:34:10.057409 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: X-Tenant: admin Oct 19 00:34:10.057409 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: X-Tenant-Id: d417ad1948a04e309383739147354388 Oct 19 00:34:10.057409 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: X-Tenant-Name: admin Oct 19 00:34:10.057409 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: X-User: admin Oct 19 00:34:10.057409 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: X-User-Domain-Id: default Oct 19 00:34:10.057409 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: X-User-Domain-Name: Default Oct 19 00:34:10.057409 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: X-User-Id: ded9b81a3e894fd88539e86bbeb17ee3 Oct 19 00:34:10.057409 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: X-User-Name: admin Oct 19 00:34:10.057409 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: method: > Oct 19 00:34:10.057409 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: exception: Flavor 42 could not be found. Oct 19 00:34:10.057409 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: action_args: {'id': '42'} Oct 19 00:34:10.057409 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: {{(pid=110597) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 19 00:34:10.057409 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 19 00:34:10.058645 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show Oct 19 00:34:10.058645 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) Oct 19 00:34:10.058645 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Oct 19 00:34:10.058645 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Oct 19 00:34:10.058645 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Oct 19 00:34:10.058645 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Oct 19 00:34:10.058645 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Oct 19 00:34:10.058645 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Oct 19 00:34:10.058645 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Oct 19 00:34:10.058645 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 19 00:34:10.058645 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Oct 19 00:34:10.058645 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Oct 19 00:34:10.058645 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 42 could not be found. Oct 19 00:34:10.058645 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: ERROR nova.api.openstack.wsgi Oct 19 00:34:10.058645 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 19 00:34:10.058645 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: ERROR nova.api.openstack.wsgi Oct 19 00:34:10.058645 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 19 00:34:10.058645 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 19 00:34:10.058645 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 19 00:34:10.060049 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 19 00:34:10.060049 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 19 00:34:10.060049 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 19 00:34:10.060049 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 19 00:34:10.060049 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show Oct 19 00:34:10.060049 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Oct 19 00:34:10.060049 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 42 could not be found. Oct 19 00:34:10.060049 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: ERROR nova.api.openstack.wsgi Oct 19 00:34:10.060049 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 19 00:34:10.060049 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: ERROR nova.api.openstack.wsgi Oct 19 00:34:10.060049 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 19 00:34:10.060049 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 19 00:34:10.060049 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 19 00:34:10.060049 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 19 00:34:10.060049 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 19 00:34:10.060049 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 42 could not be found. Oct 19 00:34:10.060049 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: ERROR nova.api.openstack.wsgi Oct 19 00:34:10.060049 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi [None req-48d9d515-0a0d-45d5-acd9-8763116ece73 admin admin] Returning 404 to user: Flavor 42 could not be found. {{(pid=110597) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 19 00:34:10.061556 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: INFO nova.api.openstack.requestlog [None req-48d9d515-0a0d-45d5-acd9-8763116ece73 admin admin] 10.222.0.26 "GET /compute/v2.1/flavors/42" status: 404 len: 75 microversion: 2.92 time: 0.484609 Oct 19 00:34:10.061556 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: [pid: 110597|app: 0|req: 19/39] 10.222.0.26 () {60 vars in 1190 bytes} [Wed Oct 19 00:34:09 2022] GET /compute/v2.1/flavors/42 => generated 75 bytes in 486 msecs (HTTP/1.1 404) 9 headers in 380 bytes (1 switches on core 0) Oct 19 00:34:10.069644 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi [None req-e5c38b54-c564-4b46-b061-35a010a68580 admin admin] Calling method '>' {{(pid=110596) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:34:10.073888 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG dbcounter [-] [110596] Writing DB stats nova_api:SELECT=8,nova_api:SAVEPOINT=1,nova_api:INSERT=2,nova_api:RELEASE=1 {{(pid=110596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 00:34:10.093923 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: INFO nova.api.openstack.requestlog [None req-e5c38b54-c564-4b46-b061-35a010a68580 admin admin] 10.222.0.26 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.92 time: 0.027818 Oct 19 00:34:10.094767 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: [pid: 110596|app: 0|req: 21/40] 10.222.0.26 () {60 vars in 1210 bytes} [Wed Oct 19 00:34:10 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 29 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 19 00:34:10.106021 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi [None req-af59c88d-b783-44e5-ba5e-df5172107f70 admin admin] Calling method '>' {{(pid=110597) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:34:10.130338 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: INFO nova.api.openstack.requestlog [None req-af59c88d-b783-44e5-ba5e-df5172107f70 admin admin] 10.222.0.26 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.92 time: 0.028984 Oct 19 00:34:10.130953 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: [pid: 110597|app: 0|req: 20/41] 10.222.0.26 () {60 vars in 1210 bytes} [Wed Oct 19 00:34:10 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 30 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 19 00:34:10.143071 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi [None req-397116be-5ae8-43be-96b5-df2cab2d35a1 admin admin] Calling method '>' {{(pid=110596) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:34:10.149307 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: INFO nova.api.openstack.wsgi [None req-397116be-5ae8-43be-96b5-df2cab2d35a1 admin admin] HTTP exception thrown: Flavor 42 could not be found. Oct 19 00:34:10.152822 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi [None req-397116be-5ae8-43be-96b5-df2cab2d35a1 admin admin] Request method failure captured: Oct 19 00:34:10.152822 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: request: GET /compute/v2.1/flavors/42 HTTP/1.1 Oct 19 00:34:10.152822 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: Accept: application/json Oct 19 00:34:10.152822 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: Accept-Encoding: gzip, deflate Oct 19 00:34:10.152822 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: Connection: keep-alive Oct 19 00:34:10.152822 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: Content-Length: 0 Oct 19 00:34:10.152822 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: Host: 10.222.0.26 Oct 19 00:34:10.152822 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: Openstack-Api-Version: compute 2.92 Oct 19 00:34:10.152822 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: Openstack-System-Scope: None Oct 19 00:34:10.152822 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: User-Agent: python-novaclient Oct 19 00:34:10.152822 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: X-Auth-Token: gAAAAABjT0YBbF7A-FWAsBy9OGnjZ1x2k_Xa4BOTENwY0c4KaGseYq8uKMG3_lWZTmtPbD6-mm3ivr7Z_KPXcIJHnozSls5OnmMjlcYNNHT2JmOBUAA3ej91OAId4mFldYMJj2YEUnHK31FipQtTj-5O9zzq8FZk-F8i-OS5X1Yp3V_GdaEhlls Oct 19 00:34:10.152822 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: X-Domain-Id: None Oct 19 00:34:10.152822 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: X-Domain-Name: None Oct 19 00:34:10.152822 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: X-Identity-Status: Confirmed Oct 19 00:34:10.152822 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: X-Is-Admin-Project: True Oct 19 00:34:10.152822 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: X-Openstack-Nova-Api-Version: 2.92 Oct 19 00:34:10.152822 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: X-Project-Domain-Id: default Oct 19 00:34:10.152822 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: X-Project-Domain-Name: Default Oct 19 00:34:10.152822 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: X-Project-Id: d417ad1948a04e309383739147354388 Oct 19 00:34:10.152822 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: X-Project-Name: admin Oct 19 00:34:10.152822 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: X-Role: admin,member,reader Oct 19 00:34:10.152822 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: X-Roles: admin,member,reader Oct 19 00:34:10.154726 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/d417ad1948a04e309383739147354388"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/d417ad1948a04e309383739147354388"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/d417ad1948a04e309383739147354388"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_d417ad1948a04e309383739147354388"}]}] Oct 19 00:34:10.154726 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: X-Tenant: admin Oct 19 00:34:10.154726 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: X-Tenant-Id: d417ad1948a04e309383739147354388 Oct 19 00:34:10.154726 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: X-Tenant-Name: admin Oct 19 00:34:10.154726 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: X-User: admin Oct 19 00:34:10.154726 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: X-User-Domain-Id: default Oct 19 00:34:10.154726 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: X-User-Domain-Name: Default Oct 19 00:34:10.154726 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: X-User-Id: ded9b81a3e894fd88539e86bbeb17ee3 Oct 19 00:34:10.154726 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: X-User-Name: admin Oct 19 00:34:10.154726 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: method: > Oct 19 00:34:10.154726 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: exception: Flavor 42 could not be found. Oct 19 00:34:10.154726 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: action_args: {'id': '42'} Oct 19 00:34:10.154726 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: {{(pid=110596) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 19 00:34:10.154726 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 19 00:34:10.158256 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show Oct 19 00:34:10.158256 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) Oct 19 00:34:10.158256 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Oct 19 00:34:10.158256 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Oct 19 00:34:10.158256 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Oct 19 00:34:10.158256 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Oct 19 00:34:10.158256 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Oct 19 00:34:10.158256 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Oct 19 00:34:10.158256 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Oct 19 00:34:10.158256 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 19 00:34:10.158256 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Oct 19 00:34:10.158256 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Oct 19 00:34:10.158256 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 42 could not be found. Oct 19 00:34:10.158256 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: ERROR nova.api.openstack.wsgi Oct 19 00:34:10.158256 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 19 00:34:10.158256 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: ERROR nova.api.openstack.wsgi Oct 19 00:34:10.158256 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 19 00:34:10.158256 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 19 00:34:10.158256 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 19 00:34:10.160063 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 19 00:34:10.160063 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 19 00:34:10.160063 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 19 00:34:10.160063 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 19 00:34:10.160063 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show Oct 19 00:34:10.160063 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Oct 19 00:34:10.160063 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 42 could not be found. Oct 19 00:34:10.160063 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: ERROR nova.api.openstack.wsgi Oct 19 00:34:10.160063 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 19 00:34:10.160063 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: ERROR nova.api.openstack.wsgi Oct 19 00:34:10.160063 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 19 00:34:10.160063 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 19 00:34:10.160063 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 19 00:34:10.160063 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 19 00:34:10.160063 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 19 00:34:10.160063 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 42 could not be found. Oct 19 00:34:10.160063 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: ERROR nova.api.openstack.wsgi Oct 19 00:34:10.160063 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi [None req-397116be-5ae8-43be-96b5-df2cab2d35a1 admin admin] Returning 404 to user: Flavor 42 could not be found. {{(pid=110596) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 19 00:34:10.161366 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: INFO nova.api.openstack.requestlog [None req-397116be-5ae8-43be-96b5-df2cab2d35a1 admin admin] 10.222.0.26 "GET /compute/v2.1/flavors/42" status: 404 len: 75 microversion: 2.92 time: 0.016257 Oct 19 00:34:10.161366 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: [pid: 110596|app: 0|req: 22/42] 10.222.0.26 () {60 vars in 1190 bytes} [Wed Oct 19 00:34:10 2022] GET /compute/v2.1/flavors/42 => generated 75 bytes in 18 msecs (HTTP/1.1 404) 9 headers in 380 bytes (1 switches on core 0) Oct 19 00:34:11.438099 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi [None req-af59c88d-b783-44e5-ba5e-df5172107f70 admin admin] Calling method '>' {{(pid=110597) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:34:11.439190 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: INFO nova.api.openstack.requestlog [None req-af59c88d-b783-44e5-ba5e-df5172107f70 admin admin] 10.222.0.26 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001591 Oct 19 00:34:11.439601 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: [pid: 110597|app: 0|req: 21/43] 10.222.0.26 () {56 vars in 1074 bytes} [Wed Oct 19 00:34:11 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 19 00:34:12.166881 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi [None req-c18a6254-74bc-4ade-85b1-89b953e03ea5 admin admin] Calling method '>' {{(pid=110596) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:34:12.177441 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: INFO nova.api.openstack.wsgi [None req-c18a6254-74bc-4ade-85b1-89b953e03ea5 admin admin] HTTP exception thrown: Flavor 84 could not be found. Oct 19 00:34:12.180851 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi [None req-c18a6254-74bc-4ade-85b1-89b953e03ea5 admin admin] Request method failure captured: Oct 19 00:34:12.180851 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: request: GET /compute/v2.1/flavors/84 HTTP/1.1 Oct 19 00:34:12.180851 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: Accept: application/json Oct 19 00:34:12.180851 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: Accept-Encoding: gzip, deflate Oct 19 00:34:12.180851 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: Connection: keep-alive Oct 19 00:34:12.180851 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: Content-Length: 0 Oct 19 00:34:12.180851 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: Host: 10.222.0.26 Oct 19 00:34:12.180851 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: Openstack-Api-Version: compute 2.92 Oct 19 00:34:12.180851 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: Openstack-System-Scope: None Oct 19 00:34:12.180851 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: User-Agent: python-novaclient Oct 19 00:34:12.180851 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: X-Auth-Token: gAAAAABjT0YD7RYjHo58M5bpICixop7ADLaw_SJPy8-ew3J66xk3rXzjVIR--hHCoVFqIIl-5UY4-Scfzxoy0bB4ACpQMs04wccQsm3DViC4erk9Q9jp9lrG54hzG0ZLle37dKSR-hw0iKJTpRu9EAfZehKkLk3NQirVqMi76GpHByVeeJhxZxA Oct 19 00:34:12.180851 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: X-Domain-Id: None Oct 19 00:34:12.180851 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: X-Domain-Name: None Oct 19 00:34:12.180851 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: X-Identity-Status: Confirmed Oct 19 00:34:12.180851 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: X-Is-Admin-Project: True Oct 19 00:34:12.180851 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: X-Openstack-Nova-Api-Version: 2.92 Oct 19 00:34:12.180851 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: X-Project-Domain-Id: default Oct 19 00:34:12.180851 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: X-Project-Domain-Name: Default Oct 19 00:34:12.180851 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: X-Project-Id: d417ad1948a04e309383739147354388 Oct 19 00:34:12.180851 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: X-Project-Name: admin Oct 19 00:34:12.180851 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: X-Role: admin,member,reader Oct 19 00:34:12.180851 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: X-Roles: admin,member,reader Oct 19 00:34:12.183281 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/d417ad1948a04e309383739147354388"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/d417ad1948a04e309383739147354388"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/d417ad1948a04e309383739147354388"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_d417ad1948a04e309383739147354388"}]}] Oct 19 00:34:12.183281 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: X-Tenant: admin Oct 19 00:34:12.183281 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: X-Tenant-Id: d417ad1948a04e309383739147354388 Oct 19 00:34:12.183281 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: X-Tenant-Name: admin Oct 19 00:34:12.183281 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: X-User: admin Oct 19 00:34:12.183281 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: X-User-Domain-Id: default Oct 19 00:34:12.183281 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: X-User-Domain-Name: Default Oct 19 00:34:12.183281 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: X-User-Id: ded9b81a3e894fd88539e86bbeb17ee3 Oct 19 00:34:12.183281 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: X-User-Name: admin Oct 19 00:34:12.183281 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: method: > Oct 19 00:34:12.183281 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: exception: Flavor 84 could not be found. Oct 19 00:34:12.183281 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: action_args: {'id': '84'} Oct 19 00:34:12.183281 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: {{(pid=110596) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 19 00:34:12.183281 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 19 00:34:12.184922 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show Oct 19 00:34:12.184922 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) Oct 19 00:34:12.184922 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Oct 19 00:34:12.184922 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Oct 19 00:34:12.184922 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Oct 19 00:34:12.184922 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Oct 19 00:34:12.184922 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Oct 19 00:34:12.184922 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Oct 19 00:34:12.184922 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Oct 19 00:34:12.184922 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 19 00:34:12.184922 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Oct 19 00:34:12.184922 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Oct 19 00:34:12.184922 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 84 could not be found. Oct 19 00:34:12.184922 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: ERROR nova.api.openstack.wsgi Oct 19 00:34:12.184922 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 19 00:34:12.184922 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: ERROR nova.api.openstack.wsgi Oct 19 00:34:12.184922 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 19 00:34:12.184922 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 19 00:34:12.184922 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 19 00:34:12.187478 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 19 00:34:12.187478 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 19 00:34:12.187478 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 19 00:34:12.187478 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 19 00:34:12.187478 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show Oct 19 00:34:12.187478 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Oct 19 00:34:12.187478 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 84 could not be found. Oct 19 00:34:12.187478 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: ERROR nova.api.openstack.wsgi Oct 19 00:34:12.187478 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 19 00:34:12.187478 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: ERROR nova.api.openstack.wsgi Oct 19 00:34:12.187478 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 19 00:34:12.187478 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 19 00:34:12.187478 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 19 00:34:12.187478 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 19 00:34:12.187478 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 19 00:34:12.187478 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 84 could not be found. Oct 19 00:34:12.187478 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: ERROR nova.api.openstack.wsgi Oct 19 00:34:12.187478 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi [None req-c18a6254-74bc-4ade-85b1-89b953e03ea5 admin admin] Returning 404 to user: Flavor 84 could not be found. {{(pid=110596) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 19 00:34:12.190652 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: INFO nova.api.openstack.requestlog [None req-c18a6254-74bc-4ade-85b1-89b953e03ea5 admin admin] 10.222.0.26 "GET /compute/v2.1/flavors/84" status: 404 len: 75 microversion: 2.92 time: 0.600837 Oct 19 00:34:12.190652 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: [pid: 110596|app: 0|req: 23/44] 10.222.0.26 () {60 vars in 1190 bytes} [Wed Oct 19 00:34:11 2022] GET /compute/v2.1/flavors/84 => generated 75 bytes in 603 msecs (HTTP/1.1 404) 9 headers in 380 bytes (2 switches on core 0) Oct 19 00:34:12.197304 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi [None req-adcd808c-78a6-4444-a80d-723c27d64d89 admin admin] Calling method '>' {{(pid=110597) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:34:12.214774 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: INFO nova.api.openstack.requestlog [None req-adcd808c-78a6-4444-a80d-723c27d64d89 admin admin] 10.222.0.26 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.92 time: 0.021004 Oct 19 00:34:12.215327 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: [pid: 110597|app: 0|req: 22/45] 10.222.0.26 () {60 vars in 1210 bytes} [Wed Oct 19 00:34:12 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 22 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 19 00:34:12.225854 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi [None req-73c28152-8ea4-464e-9acb-a21e53e07804 admin admin] Calling method '>' {{(pid=110596) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:34:12.243035 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: INFO nova.api.openstack.requestlog [None req-73c28152-8ea4-464e-9acb-a21e53e07804 admin admin] 10.222.0.26 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.92 time: 0.021095 Oct 19 00:34:12.243596 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: [pid: 110596|app: 0|req: 24/46] 10.222.0.26 () {60 vars in 1210 bytes} [Wed Oct 19 00:34:12 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 22 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 19 00:34:12.253838 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi [None req-c6bbbb15-7e23-4e83-be61-c4662f819f2f admin admin] Calling method '>' {{(pid=110597) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:34:12.259174 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: INFO nova.api.openstack.wsgi [None req-c6bbbb15-7e23-4e83-be61-c4662f819f2f admin admin] HTTP exception thrown: Flavor 84 could not be found. Oct 19 00:34:12.260715 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi [None req-c6bbbb15-7e23-4e83-be61-c4662f819f2f admin admin] Request method failure captured: Oct 19 00:34:12.260715 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: request: GET /compute/v2.1/flavors/84 HTTP/1.1 Oct 19 00:34:12.260715 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: Accept: application/json Oct 19 00:34:12.260715 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: Accept-Encoding: gzip, deflate Oct 19 00:34:12.260715 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: Connection: keep-alive Oct 19 00:34:12.260715 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: Content-Length: 0 Oct 19 00:34:12.260715 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: Host: 10.222.0.26 Oct 19 00:34:12.260715 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: Openstack-Api-Version: compute 2.92 Oct 19 00:34:12.260715 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: Openstack-System-Scope: None Oct 19 00:34:12.260715 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: User-Agent: python-novaclient Oct 19 00:34:12.260715 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: X-Auth-Token: gAAAAABjT0YD7RYjHo58M5bpICixop7ADLaw_SJPy8-ew3J66xk3rXzjVIR--hHCoVFqIIl-5UY4-Scfzxoy0bB4ACpQMs04wccQsm3DViC4erk9Q9jp9lrG54hzG0ZLle37dKSR-hw0iKJTpRu9EAfZehKkLk3NQirVqMi76GpHByVeeJhxZxA Oct 19 00:34:12.260715 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: X-Domain-Id: None Oct 19 00:34:12.260715 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: X-Domain-Name: None Oct 19 00:34:12.260715 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: X-Identity-Status: Confirmed Oct 19 00:34:12.260715 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: X-Is-Admin-Project: True Oct 19 00:34:12.260715 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: X-Openstack-Nova-Api-Version: 2.92 Oct 19 00:34:12.260715 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: X-Project-Domain-Id: default Oct 19 00:34:12.260715 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: X-Project-Domain-Name: Default Oct 19 00:34:12.260715 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: X-Project-Id: d417ad1948a04e309383739147354388 Oct 19 00:34:12.260715 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: X-Project-Name: admin Oct 19 00:34:12.260715 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: X-Role: admin,member,reader Oct 19 00:34:12.260715 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: X-Roles: admin,member,reader Oct 19 00:34:12.263982 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/d417ad1948a04e309383739147354388"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/d417ad1948a04e309383739147354388"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/d417ad1948a04e309383739147354388"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_d417ad1948a04e309383739147354388"}]}] Oct 19 00:34:12.263982 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: X-Tenant: admin Oct 19 00:34:12.263982 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: X-Tenant-Id: d417ad1948a04e309383739147354388 Oct 19 00:34:12.263982 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: X-Tenant-Name: admin Oct 19 00:34:12.263982 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: X-User: admin Oct 19 00:34:12.263982 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: X-User-Domain-Id: default Oct 19 00:34:12.263982 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: X-User-Domain-Name: Default Oct 19 00:34:12.263982 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: X-User-Id: ded9b81a3e894fd88539e86bbeb17ee3 Oct 19 00:34:12.263982 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: X-User-Name: admin Oct 19 00:34:12.263982 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: method: > Oct 19 00:34:12.263982 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: exception: Flavor 84 could not be found. Oct 19 00:34:12.263982 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: action_args: {'id': '84'} Oct 19 00:34:12.263982 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: {{(pid=110597) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 19 00:34:12.263982 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 19 00:34:12.266456 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show Oct 19 00:34:12.266456 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) Oct 19 00:34:12.266456 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Oct 19 00:34:12.266456 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Oct 19 00:34:12.266456 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Oct 19 00:34:12.266456 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Oct 19 00:34:12.266456 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Oct 19 00:34:12.266456 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Oct 19 00:34:12.266456 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Oct 19 00:34:12.266456 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 19 00:34:12.266456 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Oct 19 00:34:12.266456 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Oct 19 00:34:12.266456 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 84 could not be found. Oct 19 00:34:12.266456 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: ERROR nova.api.openstack.wsgi Oct 19 00:34:12.266456 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 19 00:34:12.266456 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: ERROR nova.api.openstack.wsgi Oct 19 00:34:12.266456 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 19 00:34:12.266456 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 19 00:34:12.266456 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 19 00:34:12.268817 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 19 00:34:12.268817 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 19 00:34:12.268817 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 19 00:34:12.268817 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 19 00:34:12.268817 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show Oct 19 00:34:12.268817 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Oct 19 00:34:12.268817 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 84 could not be found. Oct 19 00:34:12.268817 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: ERROR nova.api.openstack.wsgi Oct 19 00:34:12.268817 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 19 00:34:12.268817 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: ERROR nova.api.openstack.wsgi Oct 19 00:34:12.268817 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 19 00:34:12.268817 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 19 00:34:12.268817 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 19 00:34:12.268817 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 19 00:34:12.268817 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 19 00:34:12.268817 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 84 could not be found. Oct 19 00:34:12.268817 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: ERROR nova.api.openstack.wsgi Oct 19 00:34:12.268817 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi [None req-c6bbbb15-7e23-4e83-be61-c4662f819f2f admin admin] Returning 404 to user: Flavor 84 could not be found. {{(pid=110597) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 19 00:34:12.270680 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: INFO nova.api.openstack.requestlog [None req-c6bbbb15-7e23-4e83-be61-c4662f819f2f admin admin] 10.222.0.26 "GET /compute/v2.1/flavors/84" status: 404 len: 75 microversion: 2.92 time: 0.012074 Oct 19 00:34:12.270680 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: [pid: 110597|app: 0|req: 23/47] 10.222.0.26 () {60 vars in 1190 bytes} [Wed Oct 19 00:34:12 2022] GET /compute/v2.1/flavors/84 => generated 75 bytes in 13 msecs (HTTP/1.1 404) 9 headers in 380 bytes (1 switches on core 0) Oct 19 00:34:13.618368 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi [None req-73c28152-8ea4-464e-9acb-a21e53e07804 admin admin] Calling method '>' {{(pid=110596) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:34:13.619326 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: INFO nova.api.openstack.requestlog [None req-73c28152-8ea4-464e-9acb-a21e53e07804 admin admin] 10.222.0.26 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001480 Oct 19 00:34:13.619751 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: [pid: 110596|app: 0|req: 25/48] 10.222.0.26 () {56 vars in 1074 bytes} [Wed Oct 19 00:34:13 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 19 00:34:14.205342 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi [None req-b8d7a01f-23a1-4dc3-98fe-706561557d14 admin admin] Calling method '>' {{(pid=110597) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:34:14.211801 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: INFO nova.api.openstack.wsgi [None req-b8d7a01f-23a1-4dc3-98fe-706561557d14 admin admin] HTTP exception thrown: Flavor 451 could not be found. Oct 19 00:34:14.213344 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi [None req-b8d7a01f-23a1-4dc3-98fe-706561557d14 admin admin] Request method failure captured: Oct 19 00:34:14.213344 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: request: GET /compute/v2.1/flavors/451 HTTP/1.1 Oct 19 00:34:14.213344 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: Accept: application/json Oct 19 00:34:14.213344 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: Accept-Encoding: gzip, deflate Oct 19 00:34:14.213344 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: Connection: keep-alive Oct 19 00:34:14.213344 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: Content-Length: 0 Oct 19 00:34:14.213344 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: Host: 10.222.0.26 Oct 19 00:34:14.213344 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: Openstack-Api-Version: compute 2.92 Oct 19 00:34:14.213344 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: Openstack-System-Scope: None Oct 19 00:34:14.213344 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: User-Agent: python-novaclient Oct 19 00:34:14.213344 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: X-Auth-Token: gAAAAABjT0YFb7PKmaFbTz1397SGBdEuYlQD7Wa23fSe5bcW1ag2FiTt5430dmxXvVqPQQL7hoQoIdjNUos-nExvjyT7e0j-lwgNULd1hcLqz-e1hEUaELMnHo7L_vUVAPkr4HsSLZN7krtK3HZv-qI81bPE66YHlN4k9A6FenEawaSO0yB-C68 Oct 19 00:34:14.213344 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: X-Domain-Id: None Oct 19 00:34:14.213344 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: X-Domain-Name: None Oct 19 00:34:14.213344 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: X-Identity-Status: Confirmed Oct 19 00:34:14.213344 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: X-Is-Admin-Project: True Oct 19 00:34:14.213344 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: X-Openstack-Nova-Api-Version: 2.92 Oct 19 00:34:14.213344 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: X-Project-Domain-Id: default Oct 19 00:34:14.213344 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: X-Project-Domain-Name: Default Oct 19 00:34:14.213344 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: X-Project-Id: d417ad1948a04e309383739147354388 Oct 19 00:34:14.213344 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: X-Project-Name: admin Oct 19 00:34:14.213344 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: X-Role: admin,member,reader Oct 19 00:34:14.213344 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: X-Roles: admin,member,reader Oct 19 00:34:14.215612 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/d417ad1948a04e309383739147354388"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/d417ad1948a04e309383739147354388"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/d417ad1948a04e309383739147354388"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_d417ad1948a04e309383739147354388"}]}] Oct 19 00:34:14.215612 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: X-Tenant: admin Oct 19 00:34:14.215612 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: X-Tenant-Id: d417ad1948a04e309383739147354388 Oct 19 00:34:14.215612 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: X-Tenant-Name: admin Oct 19 00:34:14.215612 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: X-User: admin Oct 19 00:34:14.215612 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: X-User-Domain-Id: default Oct 19 00:34:14.215612 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: X-User-Domain-Name: Default Oct 19 00:34:14.215612 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: X-User-Id: ded9b81a3e894fd88539e86bbeb17ee3 Oct 19 00:34:14.215612 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: X-User-Name: admin Oct 19 00:34:14.215612 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: method: > Oct 19 00:34:14.215612 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: exception: Flavor 451 could not be found. Oct 19 00:34:14.215612 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: action_args: {'id': '451'} Oct 19 00:34:14.215612 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: {{(pid=110597) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 19 00:34:14.215612 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 19 00:34:14.217142 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show Oct 19 00:34:14.217142 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) Oct 19 00:34:14.217142 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Oct 19 00:34:14.217142 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Oct 19 00:34:14.217142 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Oct 19 00:34:14.217142 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Oct 19 00:34:14.217142 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Oct 19 00:34:14.217142 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Oct 19 00:34:14.217142 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Oct 19 00:34:14.217142 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 19 00:34:14.217142 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Oct 19 00:34:14.217142 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Oct 19 00:34:14.217142 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 451 could not be found. Oct 19 00:34:14.217142 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: ERROR nova.api.openstack.wsgi Oct 19 00:34:14.217142 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 19 00:34:14.217142 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: ERROR nova.api.openstack.wsgi Oct 19 00:34:14.217142 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 19 00:34:14.217142 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 19 00:34:14.217142 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 19 00:34:14.219018 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 19 00:34:14.219018 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 19 00:34:14.219018 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 19 00:34:14.219018 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 19 00:34:14.219018 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show Oct 19 00:34:14.219018 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Oct 19 00:34:14.219018 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 451 could not be found. Oct 19 00:34:14.219018 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: ERROR nova.api.openstack.wsgi Oct 19 00:34:14.219018 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 19 00:34:14.219018 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: ERROR nova.api.openstack.wsgi Oct 19 00:34:14.219018 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 19 00:34:14.219018 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 19 00:34:14.219018 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 19 00:34:14.219018 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 19 00:34:14.219018 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 19 00:34:14.219018 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 451 could not be found. Oct 19 00:34:14.219018 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: ERROR nova.api.openstack.wsgi Oct 19 00:34:14.219018 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi [None req-b8d7a01f-23a1-4dc3-98fe-706561557d14 admin admin] Returning 404 to user: Flavor 451 could not be found. {{(pid=110597) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 19 00:34:14.220691 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: INFO nova.api.openstack.requestlog [None req-b8d7a01f-23a1-4dc3-98fe-706561557d14 admin admin] 10.222.0.26 "GET /compute/v2.1/flavors/451" status: 404 len: 76 microversion: 2.92 time: 0.454005 Oct 19 00:34:14.220691 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: [pid: 110597|app: 0|req: 24/49] 10.222.0.26 () {60 vars in 1193 bytes} [Wed Oct 19 00:34:13 2022] GET /compute/v2.1/flavors/451 => generated 76 bytes in 455 msecs (HTTP/1.1 404) 9 headers in 380 bytes (1 switches on core 0) Oct 19 00:34:14.225777 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi [None req-19f0ef7f-5563-43b9-829c-3d1de8d4f3a4 admin admin] Calling method '>' {{(pid=110596) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:34:14.241699 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: INFO nova.api.openstack.requestlog [None req-19f0ef7f-5563-43b9-829c-3d1de8d4f3a4 admin admin] 10.222.0.26 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.92 time: 0.019687 Oct 19 00:34:14.242383 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: [pid: 110596|app: 0|req: 26/50] 10.222.0.26 () {60 vars in 1210 bytes} [Wed Oct 19 00:34:14 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 21 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 19 00:34:14.253676 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi [None req-3dc44cba-4768-4726-bb15-2ac30beb4769 admin admin] Calling method '>' {{(pid=110597) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:34:14.271078 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: INFO nova.api.openstack.requestlog [None req-3dc44cba-4768-4726-bb15-2ac30beb4769 admin admin] 10.222.0.26 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.92 time: 0.022024 Oct 19 00:34:14.271648 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: [pid: 110597|app: 0|req: 25/51] 10.222.0.26 () {60 vars in 1210 bytes} [Wed Oct 19 00:34:14 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 23 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 19 00:34:14.281730 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi [None req-11cf8b6e-7189-46cc-8b17-c7d2dda76e88 admin admin] Calling method '>' {{(pid=110596) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:34:14.287788 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: INFO nova.api.openstack.wsgi [None req-11cf8b6e-7189-46cc-8b17-c7d2dda76e88 admin admin] HTTP exception thrown: Flavor 451 could not be found. Oct 19 00:34:14.289799 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi [None req-11cf8b6e-7189-46cc-8b17-c7d2dda76e88 admin admin] Request method failure captured: Oct 19 00:34:14.289799 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: request: GET /compute/v2.1/flavors/451 HTTP/1.1 Oct 19 00:34:14.289799 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: Accept: application/json Oct 19 00:34:14.289799 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: Accept-Encoding: gzip, deflate Oct 19 00:34:14.289799 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: Connection: keep-alive Oct 19 00:34:14.289799 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: Content-Length: 0 Oct 19 00:34:14.289799 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: Host: 10.222.0.26 Oct 19 00:34:14.289799 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: Openstack-Api-Version: compute 2.92 Oct 19 00:34:14.289799 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: Openstack-System-Scope: None Oct 19 00:34:14.289799 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: User-Agent: python-novaclient Oct 19 00:34:14.289799 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: X-Auth-Token: gAAAAABjT0YFb7PKmaFbTz1397SGBdEuYlQD7Wa23fSe5bcW1ag2FiTt5430dmxXvVqPQQL7hoQoIdjNUos-nExvjyT7e0j-lwgNULd1hcLqz-e1hEUaELMnHo7L_vUVAPkr4HsSLZN7krtK3HZv-qI81bPE66YHlN4k9A6FenEawaSO0yB-C68 Oct 19 00:34:14.289799 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: X-Domain-Id: None Oct 19 00:34:14.289799 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: X-Domain-Name: None Oct 19 00:34:14.289799 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: X-Identity-Status: Confirmed Oct 19 00:34:14.289799 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: X-Is-Admin-Project: True Oct 19 00:34:14.289799 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: X-Openstack-Nova-Api-Version: 2.92 Oct 19 00:34:14.289799 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: X-Project-Domain-Id: default Oct 19 00:34:14.289799 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: X-Project-Domain-Name: Default Oct 19 00:34:14.289799 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: X-Project-Id: d417ad1948a04e309383739147354388 Oct 19 00:34:14.289799 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: X-Project-Name: admin Oct 19 00:34:14.289799 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: X-Role: admin,member,reader Oct 19 00:34:14.289799 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: X-Roles: admin,member,reader Oct 19 00:34:14.292324 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/d417ad1948a04e309383739147354388"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/d417ad1948a04e309383739147354388"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/d417ad1948a04e309383739147354388"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_d417ad1948a04e309383739147354388"}]}] Oct 19 00:34:14.292324 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: X-Tenant: admin Oct 19 00:34:14.292324 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: X-Tenant-Id: d417ad1948a04e309383739147354388 Oct 19 00:34:14.292324 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: X-Tenant-Name: admin Oct 19 00:34:14.292324 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: X-User: admin Oct 19 00:34:14.292324 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: X-User-Domain-Id: default Oct 19 00:34:14.292324 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: X-User-Domain-Name: Default Oct 19 00:34:14.292324 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: X-User-Id: ded9b81a3e894fd88539e86bbeb17ee3 Oct 19 00:34:14.292324 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: X-User-Name: admin Oct 19 00:34:14.292324 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: method: > Oct 19 00:34:14.292324 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: exception: Flavor 451 could not be found. Oct 19 00:34:14.292324 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: action_args: {'id': '451'} Oct 19 00:34:14.292324 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: {{(pid=110596) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 19 00:34:14.292324 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 19 00:34:14.293910 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show Oct 19 00:34:14.293910 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) Oct 19 00:34:14.293910 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Oct 19 00:34:14.293910 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Oct 19 00:34:14.293910 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Oct 19 00:34:14.293910 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Oct 19 00:34:14.293910 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Oct 19 00:34:14.293910 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Oct 19 00:34:14.293910 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Oct 19 00:34:14.293910 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 19 00:34:14.293910 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Oct 19 00:34:14.293910 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Oct 19 00:34:14.293910 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 451 could not be found. Oct 19 00:34:14.293910 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: ERROR nova.api.openstack.wsgi Oct 19 00:34:14.293910 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 19 00:34:14.293910 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: ERROR nova.api.openstack.wsgi Oct 19 00:34:14.293910 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 19 00:34:14.293910 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 19 00:34:14.293910 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 19 00:34:14.296072 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 19 00:34:14.296072 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 19 00:34:14.296072 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 19 00:34:14.296072 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 19 00:34:14.296072 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show Oct 19 00:34:14.296072 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Oct 19 00:34:14.296072 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 451 could not be found. Oct 19 00:34:14.296072 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: ERROR nova.api.openstack.wsgi Oct 19 00:34:14.296072 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 19 00:34:14.296072 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: ERROR nova.api.openstack.wsgi Oct 19 00:34:14.296072 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 19 00:34:14.296072 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 19 00:34:14.296072 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 19 00:34:14.296072 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 19 00:34:14.296072 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 19 00:34:14.296072 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 451 could not be found. Oct 19 00:34:14.296072 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: ERROR nova.api.openstack.wsgi Oct 19 00:34:14.296072 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi [None req-11cf8b6e-7189-46cc-8b17-c7d2dda76e88 admin admin] Returning 404 to user: Flavor 451 could not be found. {{(pid=110596) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 19 00:34:14.299015 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: INFO nova.api.openstack.requestlog [None req-11cf8b6e-7189-46cc-8b17-c7d2dda76e88 admin admin] 10.222.0.26 "GET /compute/v2.1/flavors/451" status: 404 len: 76 microversion: 2.92 time: 0.013286 Oct 19 00:34:14.299015 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: [pid: 110596|app: 0|req: 27/52] 10.222.0.26 () {60 vars in 1193 bytes} [Wed Oct 19 00:34:14 2022] GET /compute/v2.1/flavors/451 => generated 76 bytes in 15 msecs (HTTP/1.1 404) 9 headers in 380 bytes (1 switches on core 0) Oct 19 00:34:15.740615 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi [None req-3dc44cba-4768-4726-bb15-2ac30beb4769 admin admin] Calling method '>' {{(pid=110597) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:34:15.741685 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: INFO nova.api.openstack.requestlog [None req-3dc44cba-4768-4726-bb15-2ac30beb4769 admin admin] 10.222.0.26 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001520 Oct 19 00:34:15.742151 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: [pid: 110597|app: 0|req: 26/53] 10.222.0.26 () {56 vars in 1074 bytes} [Wed Oct 19 00:34:15 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 19 00:34:16.411269 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi [None req-65cd05b5-7ac0-463a-b16a-282269771460 admin admin] Action: 'create', calling method: >, body: {"flavor": {"name": "m1.nano", "ram": 256, "vcpus": 1, "disk": 1, "id": "42", "swap": 0, "OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0, "os-flavor-access:is_public": true}} {{(pid=110596) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 19 00:34:16.436043 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: INFO nova.api.openstack.requestlog [None req-65cd05b5-7ac0-463a-b16a-282269771460 admin admin] 10.222.0.26 "POST /compute/v2.1/flavors" status: 200 len: 404 microversion: 2.92 time: 0.536736 Oct 19 00:34:16.436599 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: [pid: 110596|app: 0|req: 28/54] 10.222.0.26 () {64 vars in 1235 bytes} [Wed Oct 19 00:34:15 2022] POST /compute/v2.1/flavors => generated 404 bytes in 538 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Oct 19 00:34:17.754466 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi [None req-3dc44cba-4768-4726-bb15-2ac30beb4769 admin admin] Calling method '>' {{(pid=110597) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:34:17.755564 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: INFO nova.api.openstack.requestlog [None req-3dc44cba-4768-4726-bb15-2ac30beb4769 admin admin] 10.222.0.26 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001478 Oct 19 00:34:17.755956 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: [pid: 110597|app: 0|req: 27/55] 10.222.0.26 () {56 vars in 1074 bytes} [Wed Oct 19 00:34:17 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 19 00:34:18.344753 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi [None req-2ac592df-d32e-4874-859e-536a725001b3 admin admin] Action: 'create', calling method: >, body: {"flavor": {"name": "m1.micro", "ram": 256, "vcpus": 1, "disk": 2, "id": "84", "swap": 0, "OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0, "os-flavor-access:is_public": true}} {{(pid=110596) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 19 00:34:18.364942 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: INFO nova.api.openstack.requestlog [None req-2ac592df-d32e-4874-859e-536a725001b3 admin admin] 10.222.0.26 "POST /compute/v2.1/flavors" status: 200 len: 405 microversion: 2.92 time: 0.465332 Oct 19 00:34:18.365499 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: [pid: 110596|app: 0|req: 29/56] 10.222.0.26 () {64 vars in 1235 bytes} [Wed Oct 19 00:34:17 2022] POST /compute/v2.1/flavors => generated 405 bytes in 467 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 19 00:34:19.644437 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi [None req-3dc44cba-4768-4726-bb15-2ac30beb4769 admin admin] Calling method '>' {{(pid=110597) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:34:19.645593 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: INFO nova.api.openstack.requestlog [None req-3dc44cba-4768-4726-bb15-2ac30beb4769 admin admin] 10.222.0.26 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001553 Oct 19 00:34:19.646224 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: [pid: 110597|app: 0|req: 28/57] 10.222.0.26 () {56 vars in 1074 bytes} [Wed Oct 19 00:34:19 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 19 00:34:20.237612 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi [None req-bbf67c4f-c6c2-4bf1-af70-d8d0bcfa680e admin admin] Action: 'create', calling method: >, body: {"flavor": {"name": "m1.heat", "ram": 512, "vcpus": 1, "disk": 1, "id": "451", "swap": 0, "OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0, "os-flavor-access:is_public": true}} {{(pid=110596) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 19 00:34:20.256962 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: INFO nova.api.openstack.requestlog [None req-bbf67c4f-c6c2-4bf1-af70-d8d0bcfa680e admin admin] 10.222.0.26 "POST /compute/v2.1/flavors" status: 200 len: 407 microversion: 2.92 time: 0.469188 Oct 19 00:34:20.257501 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: [pid: 110596|app: 0|req: 30/58] 10.222.0.26 () {64 vars in 1235 bytes} [Wed Oct 19 00:34:19 2022] POST /compute/v2.1/flavors => generated 407 bytes in 470 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 19 00:34:21.539984 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi [None req-3dc44cba-4768-4726-bb15-2ac30beb4769 admin admin] Calling method '>' {{(pid=110597) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:34:21.540996 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: INFO nova.api.openstack.requestlog [None req-3dc44cba-4768-4726-bb15-2ac30beb4769 admin admin] 10.222.0.26 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001447 Oct 19 00:34:21.541387 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: [pid: 110597|app: 0|req: 29/59] 10.222.0.26 () {56 vars in 1074 bytes} [Wed Oct 19 00:34:21 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 19 00:34:22.100051 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi [None req-b496a00f-3834-4e15-a760-0a4edd8443d9 admin admin] Action: 'create', calling method: >, body: {"flavor": {"name": "m1.heat2", "ram": 1024, "vcpus": 1, "disk": 1, "id": "452", "swap": 0, "OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0, "os-flavor-access:is_public": true}} {{(pid=110596) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 19 00:34:22.119935 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: INFO nova.api.openstack.requestlog [None req-b496a00f-3834-4e15-a760-0a4edd8443d9 admin admin] 10.222.0.26 "POST /compute/v2.1/flavors" status: 200 len: 409 microversion: 2.92 time: 0.439732 Oct 19 00:34:22.120488 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: [pid: 110596|app: 0|req: 31/60] 10.222.0.26 () {64 vars in 1235 bytes} [Wed Oct 19 00:34:21 2022] POST /compute/v2.1/flavors => generated 409 bytes in 441 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 19 00:34:23.389497 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi [None req-3dc44cba-4768-4726-bb15-2ac30beb4769 admin admin] Calling method '>' {{(pid=110597) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:34:23.390518 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: INFO nova.api.openstack.requestlog [None req-3dc44cba-4768-4726-bb15-2ac30beb4769 admin admin] 10.222.0.26 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001457 Oct 19 00:34:23.390946 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: [pid: 110597|app: 0|req: 30/61] 10.222.0.26 () {56 vars in 1074 bytes} [Wed Oct 19 00:34:23 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 19 00:34:23.960635 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi [None req-010d3c13-42d4-42eb-9be0-cf992b6a4589 admin admin] Calling method '>' {{(pid=110596) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:34:23.983862 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: INFO nova.api.openstack.requestlog [None req-010d3c13-42d4-42eb-9be0-cf992b6a4589 admin admin] 10.222.0.26 "GET /compute/v2.1/flavors/detail" status: 200 len: 5777 microversion: 2.92 time: 0.452371 Oct 19 00:34:23.984415 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: [pid: 110596|app: 0|req: 32/62] 10.222.0.26 () {60 vars in 1202 bytes} [Wed Oct 19 00:34:23 2022] GET /compute/v2.1/flavors/detail => generated 5777 bytes in 454 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 19 00:35:23.793500 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi [None req-3dc44cba-4768-4726-bb15-2ac30beb4769 admin admin] Calling method '>' {{(pid=110597) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:35:23.794460 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: INFO nova.api.openstack.requestlog [None req-3dc44cba-4768-4726-bb15-2ac30beb4769 admin admin] 10.222.0.26 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001490 Oct 19 00:35:23.794885 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: [pid: 110597|app: 0|req: 31/63] 10.222.0.26 () {54 vars in 931 bytes} [Wed Oct 19 00:35:23 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 19 00:35:23.806414 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG dbcounter [-] [110596] Writing DB stats nova_api:SELECT=26,nova_api:INSERT=4 {{(pid=110596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 00:35:24.222065 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi [None req-23752494-354a-4798-b713-e5ccec021bbd admin admin] Calling method '>' {{(pid=110596) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:35:24.243456 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: INFO nova.api.openstack.requestlog [None req-23752494-354a-4798-b713-e5ccec021bbd admin admin] 10.222.0.26 "GET /compute/v2.1/flavors/detail?is_public=True" status: 200 len: 5791 microversion: 2.61 time: 0.441405 Oct 19 00:35:24.243999 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: [pid: 110596|app: 0|req: 33/64] 10.222.0.26 () {60 vars in 1292 bytes} [Wed Oct 19 00:35:23 2022] GET /compute/v2.1/flavors/detail?is_public=True => generated 5791 bytes in 443 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 19 00:35:24.266425 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG dbcounter [-] [110597] Writing DB stats nova_api:SELECT=14 {{(pid=110597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 00:35:24.270272 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi [None req-20a793aa-5da6-4a76-a364-e7e1ffb0a8e5 admin admin] Calling method '>' {{(pid=110597) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:35:24.277852 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: INFO nova.api.openstack.requestlog [None req-20a793aa-5da6-4a76-a364-e7e1ffb0a8e5 admin admin] 10.222.0.26 "GET /compute/v2.1/flavors/42/os-extra_specs" status: 200 len: 19 microversion: 2.61 time: 0.012784 Oct 19 00:35:24.278397 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: [pid: 110597|app: 0|req: 32/65] 10.222.0.26 () {60 vars in 1283 bytes} [Wed Oct 19 00:35:24 2022] GET /compute/v2.1/flavors/42/os-extra_specs => generated 19 bytes in 14 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:35:24.290565 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi [None req-f0f12419-4d4e-4348-b90f-845a34e5de38 admin admin] Calling method '>' {{(pid=110596) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:35:24.298723 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: INFO nova.api.openstack.requestlog [None req-f0f12419-4d4e-4348-b90f-845a34e5de38 admin admin] 10.222.0.26 "GET /compute/v2.1/flavors/451/os-extra_specs" status: 200 len: 19 microversion: 2.61 time: 0.011708 Oct 19 00:35:24.299284 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: [pid: 110596|app: 0|req: 34/66] 10.222.0.26 () {60 vars in 1286 bytes} [Wed Oct 19 00:35:24 2022] GET /compute/v2.1/flavors/451/os-extra_specs => generated 19 bytes in 13 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:35:24.311948 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi [None req-9d29a5ac-6221-48f3-8ece-4143d592846e admin admin] Calling method '>' {{(pid=110597) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:35:24.319580 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: INFO nova.api.openstack.requestlog [None req-9d29a5ac-6221-48f3-8ece-4143d592846e admin admin] 10.222.0.26 "GET /compute/v2.1/flavors/452/os-extra_specs" status: 200 len: 19 microversion: 2.61 time: 0.011815 Oct 19 00:35:24.320132 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: [pid: 110597|app: 0|req: 33/67] 10.222.0.26 () {60 vars in 1286 bytes} [Wed Oct 19 00:35:24 2022] GET /compute/v2.1/flavors/452/os-extra_specs => generated 19 bytes in 13 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:35:24.332495 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi [None req-12a309b5-6a8f-4e45-87eb-b55f937c6770 admin admin] Calling method '>' {{(pid=110596) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:35:24.340127 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: INFO nova.api.openstack.requestlog [None req-12a309b5-6a8f-4e45-87eb-b55f937c6770 admin admin] 10.222.0.26 "GET /compute/v2.1/flavors/84/os-extra_specs" status: 200 len: 19 microversion: 2.61 time: 0.011227 Oct 19 00:35:24.340659 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: [pid: 110596|app: 0|req: 35/68] 10.222.0.26 () {60 vars in 1283 bytes} [Wed Oct 19 00:35:24 2022] GET /compute/v2.1/flavors/84/os-extra_specs => generated 19 bytes in 12 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:38:26.238750 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi [None req-9d29a5ac-6221-48f3-8ece-4143d592846e admin admin] Calling method '>' {{(pid=110597) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:38:26.239760 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: INFO nova.api.openstack.requestlog [None req-9d29a5ac-6221-48f3-8ece-4143d592846e admin admin] 10.222.0.26 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001492 Oct 19 00:38:26.240150 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: [pid: 110597|app: 0|req: 34/69] 10.222.0.26 () {56 vars in 1074 bytes} [Wed Oct 19 00:38:26 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 19 00:38:26.382904 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG dbcounter [-] [110596] Writing DB stats nova_api:SELECT=6 {{(pid=110596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 00:38:26.543045 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi [None req-5791dd38-4b00-4424-a96c-3acff38b298e admin admin] Calling method '>' {{(pid=110596) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:38:26.548126 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG oslo_concurrency.lockutils [None req-5791dd38-4b00-4424-a96c-3acff38b298e admin admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=110596) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:38:26.548621 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG oslo_concurrency.lockutils [None req-5791dd38-4b00-4424-a96c-3acff38b298e admin admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=110596) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:38:26.549045 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG oslo_concurrency.lockutils [None req-5791dd38-4b00-4424-a96c-3acff38b298e admin admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=110596) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:38:26.549748 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG oslo_concurrency.lockutils [None req-5791dd38-4b00-4424-a96c-3acff38b298e admin admin] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=110596) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:38:26.550191 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG oslo_concurrency.lockutils [None req-5791dd38-4b00-4424-a96c-3acff38b298e admin admin] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=110596) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:38:26.550621 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG oslo_concurrency.lockutils [None req-5791dd38-4b00-4424-a96c-3acff38b298e admin admin] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=110596) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:38:26.583497 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: INFO nova.api.openstack.requestlog [None req-5791dd38-4b00-4424-a96c-3acff38b298e admin admin] 10.222.0.26 "GET /compute/v2.1/os-services" status: 200 len: 791 microversion: 2.92 time: 0.202267 Oct 19 00:38:26.584055 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: [pid: 110596|app: 0|req: 36/70] 10.222.0.26 () {60 vars in 1193 bytes} [Wed Oct 19 00:38:26 2022] GET /compute/v2.1/os-services => generated 791 bytes in 204 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 19 00:38:27.685787 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi [None req-9d29a5ac-6221-48f3-8ece-4143d592846e admin admin] Calling method '>' {{(pid=110597) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:38:27.686836 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: INFO nova.api.openstack.requestlog [None req-9d29a5ac-6221-48f3-8ece-4143d592846e admin admin] 10.222.0.26 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001425 Oct 19 00:38:27.687227 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: [pid: 110597|app: 0|req: 35/71] 10.222.0.26 () {56 vars in 1074 bytes} [Wed Oct 19 00:38:27 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 19 00:38:27.877209 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi [None req-750f2abd-c954-4849-a4ae-f43381b83520 admin admin] Calling method '>' {{(pid=110596) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:38:27.880625 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG oslo_concurrency.lockutils [None req-750f2abd-c954-4849-a4ae-f43381b83520 admin admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=110596) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:38:27.881095 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG oslo_concurrency.lockutils [None req-750f2abd-c954-4849-a4ae-f43381b83520 admin admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=110596) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:38:27.881519 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG oslo_concurrency.lockutils [None req-750f2abd-c954-4849-a4ae-f43381b83520 admin admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=110596) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:38:27.883464 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG oslo_concurrency.lockutils [None req-750f2abd-c954-4849-a4ae-f43381b83520 admin admin] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=110596) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:38:27.883915 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG oslo_concurrency.lockutils [None req-750f2abd-c954-4849-a4ae-f43381b83520 admin admin] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=110596) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:38:27.884333 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG oslo_concurrency.lockutils [None req-750f2abd-c954-4849-a4ae-f43381b83520 admin admin] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=110596) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:38:27.904312 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: INFO nova.api.openstack.requestlog [None req-750f2abd-c954-4849-a4ae-f43381b83520 admin admin] 10.222.0.26 "GET /compute/v2.1/os-services" status: 200 len: 791 microversion: 2.92 time: 0.076312 Oct 19 00:38:27.904863 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: [pid: 110596|app: 0|req: 37/72] 10.222.0.26 () {60 vars in 1193 bytes} [Wed Oct 19 00:38:27 2022] GET /compute/v2.1/os-services => generated 791 bytes in 77 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Oct 19 00:38:39.398040 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi [None req-9d29a5ac-6221-48f3-8ece-4143d592846e admin admin] Calling method '>' {{(pid=110597) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:38:39.399092 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: INFO nova.api.openstack.requestlog [None req-9d29a5ac-6221-48f3-8ece-4143d592846e admin admin] 10.222.0.26 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.002101 Oct 19 00:38:39.399470 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: [pid: 110597|app: 0|req: 36/73] 10.222.0.26 () {56 vars in 1074 bytes} [Wed Oct 19 00:38:39 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 19 00:38:39.545500 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG dbcounter [-] [110596] Writing DB stats nova_cell0:SELECT=4 {{(pid=110596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 00:38:39.546220 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG dbcounter [-] [110596] Writing DB stats nova_cell1:SELECT=4 {{(pid=110596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 00:38:39.546852 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG dbcounter [-] [110596] Writing DB stats nova_api:SELECT=8 {{(pid=110596) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 00:38:39.597209 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi [None req-d9bf65d2-ca23-4ef4-91ef-5bcbb5aa36bf admin admin] Calling method '>' {{(pid=110596) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:38:39.601139 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG oslo_concurrency.lockutils [None req-d9bf65d2-ca23-4ef4-91ef-5bcbb5aa36bf admin admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=110596) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:38:39.601618 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG oslo_concurrency.lockutils [None req-d9bf65d2-ca23-4ef4-91ef-5bcbb5aa36bf admin admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=110596) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:38:39.602043 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG oslo_concurrency.lockutils [None req-d9bf65d2-ca23-4ef4-91ef-5bcbb5aa36bf admin admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=110596) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:38:39.603199 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG oslo_concurrency.lockutils [None req-d9bf65d2-ca23-4ef4-91ef-5bcbb5aa36bf admin admin] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=110596) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:38:39.603595 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG oslo_concurrency.lockutils [None req-d9bf65d2-ca23-4ef4-91ef-5bcbb5aa36bf admin admin] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=110596) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:38:39.604010 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG oslo_concurrency.lockutils [None req-d9bf65d2-ca23-4ef4-91ef-5bcbb5aa36bf admin admin] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=110596) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:38:39.629537 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: INFO nova.api.openstack.requestlog [None req-d9bf65d2-ca23-4ef4-91ef-5bcbb5aa36bf admin admin] 10.222.0.26 "GET /compute/v2.1/os-services" status: 200 len: 1011 microversion: 2.92 time: 0.088460 Oct 19 00:38:39.630091 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: [pid: 110596|app: 0|req: 38/74] 10.222.0.26 () {60 vars in 1193 bytes} [Wed Oct 19 00:38:39 2022] GET /compute/v2.1/os-services => generated 1011 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 19 00:38:40.717851 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG nova.api.openstack.wsgi [None req-9d29a5ac-6221-48f3-8ece-4143d592846e admin admin] Calling method '>' {{(pid=110597) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:38:40.718914 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: INFO nova.api.openstack.requestlog [None req-9d29a5ac-6221-48f3-8ece-4143d592846e admin admin] 10.222.0.26 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001446 Oct 19 00:38:40.719298 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: [pid: 110597|app: 0|req: 37/75] 10.222.0.26 () {56 vars in 1074 bytes} [Wed Oct 19 00:38:40 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 19 00:38:40.917668 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG nova.api.openstack.wsgi [None req-7133c697-b8de-46e0-a611-1b89c8db133a admin admin] Calling method '>' {{(pid=110596) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:38:40.921098 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG oslo_concurrency.lockutils [None req-7133c697-b8de-46e0-a611-1b89c8db133a admin admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=110596) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:38:40.921635 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG oslo_concurrency.lockutils [None req-7133c697-b8de-46e0-a611-1b89c8db133a admin admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=110596) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:38:40.922030 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG oslo_concurrency.lockutils [None req-7133c697-b8de-46e0-a611-1b89c8db133a admin admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=110596) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:38:40.922762 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG oslo_concurrency.lockutils [None req-7133c697-b8de-46e0-a611-1b89c8db133a admin admin] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=110596) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:38:40.923210 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG oslo_concurrency.lockutils [None req-7133c697-b8de-46e0-a611-1b89c8db133a admin admin] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=110596) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:38:40.923622 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: DEBUG oslo_concurrency.lockutils [None req-7133c697-b8de-46e0-a611-1b89c8db133a admin admin] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=110596) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:38:40.945916 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: INFO nova.api.openstack.requestlog [None req-7133c697-b8de-46e0-a611-1b89c8db133a admin admin] 10.222.0.26 "GET /compute/v2.1/os-services" status: 200 len: 1011 microversion: 2.92 time: 0.083249 Oct 19 00:38:40.946511 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: [pid: 110596|app: 0|req: 39/76] 10.222.0.26 () {60 vars in 1193 bytes} [Wed Oct 19 00:38:40 2022] GET /compute/v2.1/os-services => generated 1011 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 19 00:39:39.367730 dv-cinder-master-3-iscsi devstack@n-api.service[110595]: Wed Oct 19 00:39:39 2022 - graceful shutdown triggered... Oct 19 00:39:39.368256 dv-cinder-master-3-iscsi devstack@n-api.service[110596]: Gracefully killing worker 1 (pid: 110596)... Oct 19 00:39:39.367823 dv-cinder-master-3-iscsi systemd[1]: Stopping Devstack devstack@n-api.service... Oct 19 00:39:39.369067 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: Gracefully killing worker 2 (pid: 110597)... Oct 19 00:39:39.496601 dv-cinder-master-3-iscsi devstack@n-api.service[110597]: DEBUG dbcounter [-] [110597] Writing DB stats nova_api:SELECT=4 {{(pid=110597) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 00:39:40.368890 dv-cinder-master-3-iscsi devstack@n-api.service[110595]: worker 1 buried after 1 seconds Oct 19 00:39:40.369466 dv-cinder-master-3-iscsi devstack@n-api.service[110595]: worker 2 buried after 1 seconds Oct 19 00:39:40.369466 dv-cinder-master-3-iscsi devstack@n-api.service[110595]: goodbye to uWSGI. Oct 19 00:39:40.370672 dv-cinder-master-3-iscsi systemd[1]: devstack@n-api.service: Succeeded. Oct 19 00:39:40.371244 dv-cinder-master-3-iscsi systemd[1]: Stopped Devstack devstack@n-api.service. Oct 19 00:39:40.374017 dv-cinder-master-3-iscsi systemd[1]: Starting Devstack devstack@n-api.service... Oct 19 00:39:40.384575 dv-cinder-master-3-iscsi devstack@n-api.service[125626]: [uWSGI] getting INI configuration from /etc/nova/nova-api-uwsgi.ini Oct 19 00:39:40.387350 dv-cinder-master-3-iscsi devstack@n-api.service[125626]: *** Starting uWSGI 2.0.18-debian (64bit) on [Wed Oct 19 00:39:40 2022] *** Oct 19 00:39:40.387350 dv-cinder-master-3-iscsi devstack@n-api.service[125626]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Oct 19 00:39:40.387350 dv-cinder-master-3-iscsi devstack@n-api.service[125626]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Oct 19 00:39:40.387350 dv-cinder-master-3-iscsi devstack@n-api.service[125626]: nodename: dv-cinder-master-3-iscsi Oct 19 00:39:40.387350 dv-cinder-master-3-iscsi devstack@n-api.service[125626]: machine: x86_64 Oct 19 00:39:40.387350 dv-cinder-master-3-iscsi devstack@n-api.service[125626]: clock source: unix Oct 19 00:39:40.387350 dv-cinder-master-3-iscsi devstack@n-api.service[125626]: pcre jit disabled Oct 19 00:39:40.387350 dv-cinder-master-3-iscsi devstack@n-api.service[125626]: detected number of CPU cores: 4 Oct 19 00:39:40.387350 dv-cinder-master-3-iscsi devstack@n-api.service[125626]: current working directory: / Oct 19 00:39:40.387350 dv-cinder-master-3-iscsi devstack@n-api.service[125626]: detected binary path: /usr/bin/uwsgi-core Oct 19 00:39:40.388057 dv-cinder-master-3-iscsi devstack@n-api.service[125626]: your processes number limit is 47680 Oct 19 00:39:40.388057 dv-cinder-master-3-iscsi devstack@n-api.service[125626]: your memory page size is 4096 bytes Oct 19 00:39:40.388057 dv-cinder-master-3-iscsi devstack@n-api.service[125626]: detected max file descriptor number: 2048 Oct 19 00:39:40.388057 dv-cinder-master-3-iscsi devstack@n-api.service[125626]: lock engine: pthread robust mutexes Oct 19 00:39:40.388057 dv-cinder-master-3-iscsi devstack@n-api.service[125626]: thunder lock: enabled Oct 19 00:39:40.388057 dv-cinder-master-3-iscsi devstack@n-api.service[125626]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/nova-api-wsgi.socket fd 4 Oct 19 00:39:40.388057 dv-cinder-master-3-iscsi devstack@n-api.service[125626]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Oct 19 00:39:40.429454 dv-cinder-master-3-iscsi devstack@n-api.service[125626]: Python main interpreter initialized at 0x5615d8e729c0 Oct 19 00:39:40.429454 dv-cinder-master-3-iscsi devstack@n-api.service[125626]: python threads support enabled Oct 19 00:39:40.429454 dv-cinder-master-3-iscsi devstack@n-api.service[125626]: your server socket listen backlog is limited to 100 connections Oct 19 00:39:40.429454 dv-cinder-master-3-iscsi devstack@n-api.service[125626]: your mercy for graceful operations on workers is 90 seconds Oct 19 00:39:40.429963 dv-cinder-master-3-iscsi devstack@n-api.service[125626]: mapped 403077 bytes (393 KB) for 2 cores Oct 19 00:39:40.429963 dv-cinder-master-3-iscsi devstack@n-api.service[125626]: *** Operational MODE: preforking *** Oct 19 00:39:40.430194 dv-cinder-master-3-iscsi devstack@n-api.service[125626]: *** uWSGI is running in multiple interpreter mode *** Oct 19 00:39:40.430379 dv-cinder-master-3-iscsi devstack@n-api.service[125626]: spawned uWSGI master process (pid: 125626) Oct 19 00:39:40.430615 dv-cinder-master-3-iscsi systemd[1]: Started Devstack devstack@n-api.service. Oct 19 00:39:40.431639 dv-cinder-master-3-iscsi devstack@n-api.service[125626]: spawned uWSGI worker 1 (pid: 125640, cores: 1) Oct 19 00:39:40.431883 dv-cinder-master-3-iscsi devstack@n-api.service[125626]: spawned uWSGI worker 2 (pid: 125641, cores: 1) Oct 19 00:39:40.433400 dv-cinder-master-3-iscsi devstack@n-api.service[125626]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Oct 19 00:39:41.111991 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Modules with known eventlet monkey patching issues were imported prior to eventlet monkey patching: urllib3. This warning can usually be ignored if the caller is only importing and not executing nova code. Oct 19 00:39:41.122837 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: Modules with known eventlet monkey patching issues were imported prior to eventlet monkey patching: urllib3. This warning can usually be ignored if the caller is only importing and not executing nova code. Oct 19 00:39:42.292863 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 19 00:39:42.292863 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: from cryptography.utils import int_from_bytes Oct 19 00:39:42.293535 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 19 00:39:42.293535 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: from cryptography.utils import int_from_bytes Oct 19 00:39:42.320964 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 19 00:39:42.320964 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: from cryptography.utils import int_from_bytes Oct 19 00:39:42.321564 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 19 00:39:42.321564 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: from cryptography.utils import int_from_bytes Oct 19 00:39:44.759785 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Oct 19 00:39:44.760699 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] Configuration options gathered from: {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Oct 19 00:39:44.761240 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] command line args: [] {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Oct 19 00:39:44.761761 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Oct 19 00:39:44.762183 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] Configuration options gathered from: {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Oct 19 00:39:44.762276 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] config files: ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Oct 19 00:39:44.762276 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] ================================================================================ {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Oct 19 00:39:44.762418 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] command line args: [] {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Oct 19 00:39:44.762634 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] allow_resize_to_same_host = True {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.762798 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] config files: ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Oct 19 00:39:44.762920 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] arq_binding_timeout = 300 {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.763070 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] ================================================================================ {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Oct 19 00:39:44.763239 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries = 60 {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.763422 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] allow_resize_to_same_host = True {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.763689 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries_interval = 3 {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.763795 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] arq_binding_timeout = 300 {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.764032 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] cert = self.pem {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.764156 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries = 60 {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.764355 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] compute_driver = None {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.764531 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries_interval = 3 {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.764681 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] compute_monitors = [] {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.764846 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] cert = self.pem {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.765011 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] config_dir = [] {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.765186 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] compute_driver = None {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.765343 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] config_drive_format = iso9660 {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.765504 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] compute_monitors = [] {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.765599 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] config_file = ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.765789 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] config_dir = [] {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.765950 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] config_source = [] {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.766122 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] config_drive_format = iso9660 {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.766230 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] console_host = dv-cinder-master-3-iscsi {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.766378 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] config_file = ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.766599 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] control_exchange = nova {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.766718 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] config_source = [] {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.766921 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] cpu_allocation_ratio = None {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.767042 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] console_host = dv-cinder-master-3-iscsi {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.767241 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] daemon = False {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.767342 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] control_exchange = nova {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.767574 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] debug = True {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.767684 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] cpu_allocation_ratio = None {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.767885 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] default_access_ip_network_name = None {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.768054 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] daemon = False {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.768200 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] default_availability_zone = nova {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.768352 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] debug = True {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.768532 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] default_ephemeral_format = None {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.768699 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] default_access_ip_network_name = None {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.768846 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] default_availability_zone = nova {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.769048 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'glanceclient=WARN', 'oslo.privsep.daemon=INFO'] {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.775851 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] default_ephemeral_format = None {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.776086 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] default_schedule_zone = None {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.776256 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'glanceclient=WARN', 'oslo.privsep.daemon=INFO'] {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.776421 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] disk_allocation_ratio = None {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.776579 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] default_schedule_zone = None {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.776727 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] enable_new_services = True {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.776874 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] disk_allocation_ratio = None {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.776874 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] enable_new_services = True {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.776874 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] enabled_apis = ['osapi_compute'] {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.777304 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] enabled_apis = ['osapi_compute'] {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.777470 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] enabled_ssl_apis = [] {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.777624 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] enabled_ssl_apis = [] {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.777787 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] flat_injected = False {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.777951 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] flat_injected = False {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.778106 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] force_config_drive = True {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.778259 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] force_config_drive = True {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.778412 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] force_raw_images = True {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.778582 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] force_raw_images = True {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.778748 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] graceful_shutdown_timeout = 5 {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.778906 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] graceful_shutdown_timeout = 5 {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.779051 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] heal_instance_info_cache_interval = 60 {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.779198 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] heal_instance_info_cache_interval = 60 {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.779343 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] host = dv-cinder-master-3-iscsi {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.779497 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] host = dv-cinder-master-3-iscsi {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.779665 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] initial_cpu_allocation_ratio = 4.0 {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.779819 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] initial_cpu_allocation_ratio = 4.0 {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.779964 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] initial_disk_allocation_ratio = 1.0 {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.780114 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] initial_disk_allocation_ratio = 1.0 {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.780283 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] initial_ram_allocation_ratio = 1.0 {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.780430 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] initial_ram_allocation_ratio = 1.0 {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.780586 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] injected_network_template = /opt/stack/nova/nova/virt/interfaces.template {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.780746 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] injected_network_template = /opt/stack/nova/nova/virt/interfaces.template {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.780893 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] instance_build_timeout = 0 {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.781037 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] instance_build_timeout = 0 {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.781194 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] instance_delete_interval = 300 {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.781349 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] instance_delete_interval = 300 {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.781505 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] instance_format = [instance: %(uuid)s] {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.781673 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] instance_format = [instance: %(uuid)s] {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.781819 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] instance_name_template = instance-%08x {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.781971 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] instance_name_template = instance-%08x {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.782129 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit = False {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.782633 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit = False {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.782793 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit_period = month {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.782955 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit_period = month {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.783101 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.783255 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.783415 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] instances_path = /opt/stack/data/nova/instances {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.784214 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] instances_path = /opt/stack/data/nova/instances {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.784383 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] internal_service_availability_zone = internal {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.784556 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] internal_service_availability_zone = internal {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.784705 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] key = None {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.784864 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] key = None {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.785019 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] live_migration_retry_count = 30 {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.785165 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] live_migration_retry_count = 30 {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.785318 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] log_config_append = None {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.785466 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] log_config_append = None {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.785676 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.786359 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.786546 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] log_dir = None {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.786701 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] log_dir = None {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.786853 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] log_file = None {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.786853 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] log_options = True {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.787148 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] log_file = None {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.787308 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval = 1 {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.787462 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] log_options = True {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.787462 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval = 1 {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.787743 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval_type = days {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.787743 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] log_rotation_type = none {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.788025 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval_type = days {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.788181 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.788331 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] log_rotation_type = none {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.788484 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.788639 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.788799 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.789039 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.789188 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.789348 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.789502 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.789651 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.789810 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] long_rpc_timeout = 1800 {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.789963 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.790118 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_builds = 10 {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.790263 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] long_rpc_timeout = 1800 {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.790415 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_live_migrations = 1 {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.790598 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_builds = 10 {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.790770 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_snapshots = 5 {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.790916 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_live_migrations = 1 {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.791069 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] max_local_block_devices = 3 {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.791220 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_snapshots = 5 {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.791373 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_count = 30 {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.791531 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] max_local_block_devices = 3 {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.791684 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_size_mb = 200 {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.791840 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_count = 30 {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.791992 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] maximum_instance_delete_attempts = 5 {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.792143 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_size_mb = 200 {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.792295 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen = 0.0.0.0 {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.792466 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] maximum_instance_delete_attempts = 5 {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.792772 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen_port = 8775 {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.792940 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen = 0.0.0.0 {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.793087 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] metadata_workers = 2 {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.793232 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen_port = 8775 {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.793388 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] migrate_max_retries = -1 {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.793543 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] metadata_workers = 2 {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.793695 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] mkisofs_cmd = genisoimage {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.793849 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] migrate_max_retries = -1 {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.793994 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] my_block_storage_ip = 10.222.0.26 {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.793994 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] my_ip = 10.222.0.26 {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.794517 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] mkisofs_cmd = genisoimage {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.794692 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] network_allocate_retries = 0 {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.794854 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] my_block_storage_ip = 10.222.0.26 {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.795018 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] non_inheritable_image_properties = ['cache_in_nova', 'bittorrent'] {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.795172 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] my_ip = 10.222.0.26 {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.795325 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen = 0.0.0.0 {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.795471 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] network_allocate_retries = 0 {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.795625 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen_port = 8774 {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.795771 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] non_inheritable_image_properties = ['cache_in_nova', 'bittorrent'] {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.795924 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_unique_server_name_scope = {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.796077 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen = 0.0.0.0 {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.796230 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_workers = 2 {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.796389 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen_port = 8774 {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.796535 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] password_length = 12 {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.796686 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_unique_server_name_scope = {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.796840 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] periodic_enable = True {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.796997 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_workers = 2 {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.797143 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] periodic_fuzzy_delay = 60 {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.797301 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] password_length = 12 {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.797455 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] pointer_model = usbtablet {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.797601 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] periodic_enable = True {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.797752 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] preallocate_images = none {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.797898 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] periodic_fuzzy_delay = 60 {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.798043 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] publish_errors = False {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.798195 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] pointer_model = usbtablet {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.798347 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] pybasedir = /opt/stack/nova {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.798517 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] preallocate_images = none {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.798681 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] ram_allocation_ratio = None {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.798844 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] publish_errors = False {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.799006 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_burst = 0 {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.799161 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] pybasedir = /opt/stack/nova {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.799315 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_except_level = CRITICAL {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.799468 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] ram_allocation_ratio = None {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.799621 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_interval = 0 {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.799774 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_burst = 0 {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.799935 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] reboot_timeout = 0 {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.800081 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_except_level = CRITICAL {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.800226 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] reclaim_instance_interval = 0 {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.800370 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_interval = 0 {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.800522 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] record = None {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.800676 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] reboot_timeout = 0 {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.800836 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] reimage_timeout_per_gb = 20 {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.800998 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] reclaim_instance_interval = 0 {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.801149 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] report_interval = 10 {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.801341 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] record = None {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.801496 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] rescue_timeout = 0 {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.801647 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] reimage_timeout_per_gb = 20 {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.801808 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_cpus = 0 {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.801961 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] report_interval = 10 {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.802106 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_disk_mb = 0 {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.802252 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] rescue_timeout = 0 {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.802403 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_memory_mb = 512 {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.802583 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_cpus = 0 {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.802919 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] reserved_huge_pages = None {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.803074 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_disk_mb = 0 {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.803231 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] resize_confirm_window = 0 {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.803378 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_memory_mb = 512 {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.803532 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] resize_fs_using_block_device = False {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.803686 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] reserved_huge_pages = None {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.803846 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] resume_guests_state_on_host_boot = False {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.804000 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] resize_confirm_window = 0 {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.804152 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] rootwrap_config = /etc/nova/rootwrap.conf {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.804407 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] resize_fs_using_block_device = False {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.804667 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_action = reap {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.804831 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] resume_guests_state_on_host_boot = False {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.804985 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_poll_interval = 1800 {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.805139 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] rootwrap_config = /etc/nova/rootwrap.conf {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.805295 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_timeout = 0 {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.805458 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_action = reap {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.805618 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] scheduler_instance_sync_interval = 120 {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.805765 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_poll_interval = 1800 {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.805917 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] service_down_time = 60 {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.806062 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_timeout = 0 {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.806216 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] servicegroup_driver = db {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.806361 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] scheduler_instance_sync_interval = 120 {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.806527 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] shelved_offload_time = 0 {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.806695 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] service_down_time = 60 {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.806841 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] shelved_poll_interval = 3600 {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.806985 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] servicegroup_driver = db {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.807137 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] shutdown_timeout = 0 {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.807300 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] shelved_offload_time = 0 {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.807447 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] source_is_ipv6 = False {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.807607 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] shelved_poll_interval = 3600 {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.807760 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] ssl_only = False {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.807906 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] shutdown_timeout = 0 {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.808057 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] state_path = /opt/stack/data/nova {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.808210 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] source_is_ipv6 = False {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.808355 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_interval = 600 {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.808509 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] ssl_only = False {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.808655 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_pool_size = 1000 {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.808800 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] state_path = /opt/stack/data/nova {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.808944 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] syslog_log_facility = LOG_USER {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.809098 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_interval = 600 {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.809254 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] tempdir = None {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.809254 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] timeout_nbd = 10 {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.809547 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_pool_size = 1000 {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.809704 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] transport_url = **** {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.809857 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] syslog_log_facility = LOG_USER {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.810015 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] update_resources_interval = 0 {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.810170 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] tempdir = None {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.810326 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] use_cow_images = True {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.810503 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] timeout_nbd = 10 {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.810669 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] use_eventlog = False {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.810861 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] transport_url = **** {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.811010 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] use_journal = False {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.811177 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] update_resources_interval = 0 {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.811333 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] use_json = False {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.811962 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] use_cow_images = True {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.812128 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] use_rootwrap_daemon = False {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.812275 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] use_eventlog = False {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.812429 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] use_stderr = False {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.812594 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] use_journal = False {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.812752 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] use_syslog = False {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.813067 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] use_json = False {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.813221 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] vcpu_pin_set = None {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.813378 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] use_rootwrap_daemon = False {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.813532 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_is_fatal = True {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.813698 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] use_stderr = False {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.813847 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_timeout = 300 {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.814012 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] use_syslog = False {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.814166 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] virt_mkfs = [] {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.814319 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] vcpu_pin_set = None {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.814473 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] volume_usage_poll_interval = 0 {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.814676 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_is_fatal = True {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.814935 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] watch_log_file = False {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.815183 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_timeout = 300 {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.815348 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] web = /usr/share/spice-html5 {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.815497 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] virt_mkfs = [] {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.815651 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.disable_process_locking = False {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.815798 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] volume_usage_poll_interval = 0 {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.815943 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.lock_path = /opt/stack/data/nova {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.816108 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] watch_log_file = False {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.816262 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] api.auth_strategy = keystone {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.816408 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] web = /usr/share/spice-html5 {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 19 00:39:44.816563 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] api.compute_link_prefix = None {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.817867 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.disable_process_locking = False {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.818034 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] api.config_drive_skip_versions = 1.0 2007-01-19 2007-03-01 2007-08-29 2007-10-10 2007-12-15 2008-02-01 2008-09-01 {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.818186 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.lock_path = /opt/stack/data/nova {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.818339 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] api.dhcp_domain = novalocal {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.818613 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] api.auth_strategy = keystone {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.818779 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] api.enable_instance_password = True {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.818933 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] api.compute_link_prefix = None {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.819178 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] api.glance_link_prefix = None {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.819328 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] api.config_drive_skip_versions = 1.0 2007-01-19 2007-03-01 2007-08-29 2007-10-10 2007-12-15 2008-02-01 2008-09-01 {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.819475 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_fixed_size = 100 {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.819620 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] api.dhcp_domain = novalocal {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.819765 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_strategy = distributed {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.819910 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] api.enable_instance_password = True {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.820070 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_per_project_cells = False {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.820225 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] api.glance_link_prefix = None {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.820379 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] api.list_records_by_skipping_down_cells = True {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.820533 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_fixed_size = 100 {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.820689 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] api.local_metadata_per_cell = False {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.820689 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] api.max_limit = 1000 {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.820996 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_strategy = distributed {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.821152 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] api.metadata_cache_expiration = 15 {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.821305 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_per_project_cells = False {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.821305 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] api.list_records_by_skipping_down_cells = True {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.821599 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] api.neutron_default_tenant_id = default {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.821754 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] api.local_metadata_per_cell = False {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.821910 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] api.use_forwarded_for = False {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.821910 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] api.use_neutron_default_nets = False {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.822196 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] api.max_limit = 1000 {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.822349 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_connect_timeout = 5 {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.822524 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] api.metadata_cache_expiration = 15 {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.822677 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_failure_fatal = False {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.822836 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] api.neutron_default_tenant_id = default {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.823020 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_read_timeout = 5 {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.823385 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] api.use_forwarded_for = False {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.823533 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_ssl_certfile = {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.823679 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] api.use_neutron_default_nets = False {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.823824 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_targets = [] {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.823981 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_connect_timeout = 5 {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.824135 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_jsonfile_path = None {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.824290 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_failure_fatal = False {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.824443 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_providers = ['StaticJSON'] {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.824590 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_read_timeout = 5 {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.824590 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_ssl_certfile = {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.824877 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend = dogpile.cache.memcached {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.824877 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend_argument = **** {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.825437 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_targets = [] {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.825437 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_jsonfile_path = None {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.825728 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] cache.config_prefix = cache.oslo {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.825728 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] cache.dead_timeout = 60.0 {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.826029 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_providers = ['StaticJSON'] {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.826184 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] cache.debug_cache_backend = False {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.826338 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend = dogpile.cache.memcached {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.826502 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_retry_client = False {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.826655 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend_argument = **** {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.826818 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_socket_keepalive = False {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.826972 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] cache.config_prefix = cache.oslo {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.827125 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] cache.enabled = True {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.827125 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] cache.expiration_time = 600 {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.827419 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] cache.dead_timeout = 60.0 {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.827564 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_attempts = 2 {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.827719 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] cache.debug_cache_backend = False {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.827864 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_delay = 1.0 {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.828015 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_retry_client = False {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.828168 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_dead_retry = 300 {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.828322 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_socket_keepalive = False {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.828468 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_password = {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.828627 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] cache.enabled = True {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.828627 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] cache.expiration_time = 600 {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.828909 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.829064 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_attempts = 2 {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.829220 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.829376 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_delay = 1.0 {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.829522 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_maxsize = 10 {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.829668 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_dead_retry = 300 {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.829814 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_unused_timeout = 60 {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.829958 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_password = {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.830112 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_sasl_enabled = False {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.830268 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.830420 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_servers = ['localhost:11211'] {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.830647 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.830807 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_socket_timeout = 1.0 {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.830954 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_maxsize = 10 {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.831106 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_username = {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.831337 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_unused_timeout = 60 {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.831534 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] cache.proxies = [] {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.831684 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_sasl_enabled = False {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.831841 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_attempts = 2 {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.832051 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_servers = ['localhost:11211'] {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.832216 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_delay = 0.0 {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.832424 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_socket_timeout = 1.0 {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.832660 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_count = 1 {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.832892 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_username = {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.833052 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_idle = 1 {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.834093 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] cache.proxies = [] {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.834269 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_interval = 1 {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.834444 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_attempts = 2 {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.834621 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_allowed_ciphers = None {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.834790 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_delay = 0.0 {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.834951 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_cafile = None {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.835103 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_count = 1 {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.835287 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_certfile = None {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.835467 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_idle = 1 {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.835622 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_enabled = False {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.835775 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_interval = 1 {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.835939 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_keyfile = None {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.836093 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_allowed_ciphers = None {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.836264 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_section = None {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.836421 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_cafile = None {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.836567 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_type = password {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.836666 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_certfile = None {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.837003 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cafile = None {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.837157 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_enabled = False {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.837310 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] cinder.catalog_info = volumev3::publicURL {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.837433 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_keyfile = None {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.837541 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] cinder.certfile = None {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.837689 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_section = None {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.837843 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] cinder.collect_timing = False {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.837981 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_type = password {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.838238 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cross_az_attach = True {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.838394 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cafile = None {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.838740 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] cinder.debug = False {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.838896 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] cinder.catalog_info = volumev3::publicURL {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.838994 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] cinder.endpoint_template = None {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.839098 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] cinder.certfile = None {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.839188 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] cinder.http_retries = 3 {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.839283 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] cinder.collect_timing = False {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.839400 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] cinder.insecure = False {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.839400 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] cinder.keyfile = None {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.839670 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cross_az_attach = True {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.839820 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] cinder.os_region_name = RegionOne {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.840068 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] cinder.debug = False {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.840068 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] cinder.endpoint_template = None {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.840345 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] cinder.split_loggers = False {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.840345 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] cinder.timeout = None {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.840554 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] cinder.http_retries = 3 {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.840554 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] cinder.insecure = False {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.840759 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] compute.consecutive_build_service_disable_threshold = 10 {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.840759 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_dedicated_set = None {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.840925 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] cinder.keyfile = None {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.841013 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_shared_set = None {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.841106 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] cinder.os_region_name = RegionOne {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.841205 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] compute.image_type_exclude_list = [] {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.841370 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] cinder.split_loggers = False {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.841560 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] compute.live_migration_wait_for_vif_plug = True {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.841725 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] cinder.timeout = None {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.841873 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_concurrent_disk_ops = 0 {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.842025 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] compute.consecutive_build_service_disable_threshold = 10 {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.842224 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_disk_devices_to_attach = -1 {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.842391 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_dedicated_set = None {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.842552 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] compute.packing_host_numa_cells_allocation_strategy = False {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.842694 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_shared_set = None {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.842854 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] compute.provider_config_location = /etc/nova/provider_config/ {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.843005 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] compute.image_type_exclude_list = [] {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.843005 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] compute.live_migration_wait_for_vif_plug = True {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.843263 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] compute.resource_provider_association_refresh = 300 {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.843406 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_concurrent_disk_ops = 0 {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.843542 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] compute.shutdown_retry_interval = 10 {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.843542 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] conductor.workers = 2 {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.843797 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_disk_devices_to_attach = -1 {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.843946 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] console.allowed_origins = [] {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.844182 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] compute.packing_host_numa_cells_allocation_strategy = False {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.844317 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_ciphers = None {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.844457 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] compute.provider_config_location = /etc/nova/provider_config/ {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.844595 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_minimum_version = default {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.844736 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] compute.resource_provider_association_refresh = 300 {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.844868 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] consoleauth.token_ttl = 600 {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.844998 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] compute.shutdown_retry_interval = 10 {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.845128 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.cafile = None {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.845268 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] conductor.workers = 2 {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.845409 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.certfile = None {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.845409 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.collect_timing = False {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.845925 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] console.allowed_origins = [] {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.846060 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retries = None {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.846180 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_ciphers = None {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.846180 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_minimum_version = default {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.846570 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retry_delay = None {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.846570 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.endpoint_override = None {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.846823 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] consoleauth.token_ttl = 600 {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.846967 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.insecure = False {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.847113 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.cafile = None {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.847246 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.keyfile = None {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.847387 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.certfile = None {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.847519 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.max_version = None {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.847657 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.collect_timing = False {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.847797 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.min_version = None {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.847937 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retries = None {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.848101 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.region_name = None {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.848245 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retry_delay = None {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.848245 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.endpoint_override = None {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.848510 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_name = None {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.848787 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.insecure = False {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.848927 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_type = accelerator {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.849070 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.keyfile = None {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.849202 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.split_loggers = False {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.849202 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retries = None {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.849459 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.max_version = None {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.849591 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retry_delay = None {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.849729 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.min_version = None {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.849870 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.timeout = None {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.850009 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.region_name = None {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.850009 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_name = None {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.850264 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.valid_interfaces = ['internal', 'public'] {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.850395 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_type = accelerator {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.850553 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.version = None {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.850698 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.split_loggers = False {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.850837 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] database.backend = sqlalchemy {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.850968 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retries = None {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.851106 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] database.connection = **** {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.851244 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retry_delay = None {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.851375 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_debug = 0 {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.851520 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.timeout = None {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.851651 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_parameters = {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.851781 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.valid_interfaces = ['internal', 'public'] {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.851911 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_recycle_time = 3600 {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.852041 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.version = None {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.852184 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_trace = False {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.852323 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] database.backend = sqlalchemy {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.852461 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] database.db_inc_retry_interval = True {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.852601 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] database.connection = **** {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.852732 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retries = 20 {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.852869 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_debug = 0 {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.853014 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retry_interval = 10 {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.853014 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] database.db_retry_interval = 1 {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.853261 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_parameters = {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.853391 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] database.max_overflow = 50 {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.853528 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_recycle_time = 3600 {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.853658 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] database.max_pool_size = 5 {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.853803 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_trace = False {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.853942 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] database.max_retries = 10 {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.854071 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] database.db_inc_retry_interval = True {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.854284 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_enable_ndb = False {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.854404 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retries = 20 {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.854404 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retry_interval = 10 {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.854747 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_sql_mode = TRADITIONAL {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.854747 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] database.pool_timeout = None {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.854962 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] database.db_retry_interval = 1 {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.855100 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] database.retry_interval = 10 {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.855239 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] database.max_overflow = 50 {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.855378 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] database.slave_connection = **** {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.855545 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] database.max_pool_size = 5 {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.855694 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] database.sqlite_synchronous = True {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.855840 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] database.max_retries = 10 {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.855985 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] api_database.backend = sqlalchemy {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.856138 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_enable_ndb = False {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.856293 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection = **** {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.856454 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_sql_mode = TRADITIONAL {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.856599 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_debug = 0 {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.856753 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] database.pool_timeout = None {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.856899 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_parameters = {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.857058 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] database.retry_interval = 10 {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.857212 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_recycle_time = 3600 {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.857365 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] database.slave_connection = **** {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.857519 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_trace = False {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.857665 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] database.sqlite_synchronous = True {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.857818 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_inc_retry_interval = True {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.857972 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] api_database.backend = sqlalchemy {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.858126 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retries = 20 {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.858285 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection = **** {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.858442 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retry_interval = 10 {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.858613 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_debug = 0 {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.858759 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_retry_interval = 1 {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.858903 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_parameters = {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.859048 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_overflow = 50 {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.859180 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_recycle_time = 3600 {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.859309 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_pool_size = 5 {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.859447 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_trace = False {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.859578 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_retries = 10 {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.859706 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_inc_retry_interval = True {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.859819 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_enable_ndb = False {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.859957 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retries = 20 {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.860089 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_sql_mode = TRADITIONAL {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.860220 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retry_interval = 10 {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.860350 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] api_database.pool_timeout = None {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.860493 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_retry_interval = 1 {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.860613 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] api_database.retry_interval = 10 {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.860724 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_overflow = 50 {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.860853 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] api_database.slave_connection = **** {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.860985 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_pool_size = 5 {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.861115 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] api_database.sqlite_synchronous = True {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.861264 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_retries = 10 {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.861404 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] devices.enabled_mdev_types = [] {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.861542 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_enable_ndb = False {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.861542 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_sql_mode = TRADITIONAL {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.861542 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] api_database.pool_timeout = None {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.861542 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] api_database.retry_interval = 10 {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.862031 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.cipher = aes-xts-plain64 {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.862163 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] api_database.slave_connection = **** {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.862300 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.enabled = False {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.862439 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] api_database.sqlite_synchronous = True {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.862596 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.key_size = 512 {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.862729 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] devices.enabled_mdev_types = [] {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.862869 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] glance.api_servers = None {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.863008 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.cipher = aes-xts-plain64 {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.863140 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] glance.cafile = None {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.863269 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.enabled = False {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.863407 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] glance.certfile = None {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.863546 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.key_size = 512 {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.863720 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] glance.collect_timing = False {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.863857 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] glance.api_servers = None {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.864135 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retries = None {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.864290 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] glance.cafile = None {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.864500 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retry_delay = None {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.864500 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] glance.debug = False {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.864500 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] glance.default_trusted_certificate_ids = [] {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.864500 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_certificate_validation = False {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.864500 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_rbd_download = False {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.865139 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] glance.certfile = None {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.865271 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] glance.endpoint_override = None {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.865411 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] glance.collect_timing = False {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.865551 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] glance.insecure = False {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.865683 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retries = None {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.865813 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] glance.keyfile = None {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.865954 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retry_delay = None {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.866094 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] glance.max_version = None {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.866236 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] glance.debug = False {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.866375 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] glance.min_version = None {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.866554 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] glance.default_trusted_certificate_ids = [] {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.866688 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] glance.num_retries = 3 {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.866812 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_certificate_validation = False {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.866944 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_ceph_conf = {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.867075 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_rbd_download = False {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.867215 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_connect_timeout = 5 {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.867347 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] glance.endpoint_override = None {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.867484 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_pool = {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.867615 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] glance.insecure = False {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.867615 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] glance.keyfile = None {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.867615 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] glance.max_version = None {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.867615 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] glance.min_version = None {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.868103 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_user = {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.868103 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] glance.region_name = None {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.868103 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_name = None {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.868489 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] glance.num_retries = 3 {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.868622 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_type = image {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.868751 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_ceph_conf = {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.868888 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] glance.split_loggers = False {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.869018 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_connect_timeout = 5 {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.869156 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retries = None {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.869297 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_pool = {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.869430 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retry_delay = None {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.869559 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_user = {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.869696 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] glance.timeout = None {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.869836 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] glance.region_name = None {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.869976 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] glance.valid_interfaces = ['internal', 'public'] {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.870107 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_name = None {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.870236 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] glance.verify_glance_signatures = False {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.870378 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_type = image {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.870524 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] glance.version = None {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.870668 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] glance.split_loggers = False {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.870813 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] guestfs.debug = False {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.870953 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retries = None {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.870953 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retry_delay = None {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.871208 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_cdrom = False {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.871338 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] glance.timeout = None {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.871468 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_inject_password = False {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.871468 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.dynamic_memory_ratio = 1.0 {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.871729 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] glance.valid_interfaces = ['internal', 'public'] {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.871868 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_instance_metrics_collection = False {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.872005 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] glance.verify_glance_signatures = False {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.872145 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_remotefx = False {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.872275 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] glance.version = None {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.872484 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.instances_path_share = {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.872618 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] guestfs.debug = False {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.872756 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.iscsi_initiator_list = [] {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.872887 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_cdrom = False {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.873017 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.limit_cpu_features = False {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.873156 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_inject_password = False {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.873295 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_count = 10 {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.873434 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.dynamic_memory_ratio = 1.0 {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.873575 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_interval = 5 {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.873729 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_instance_metrics_collection = False {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.873880 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_check_timeframe = 60 {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.874022 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_remotefx = False {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.874153 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_event_polling_interval = 2 {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.874291 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.instances_path_share = {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.874422 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.qemu_img_cmd = qemu-img.exe {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.874648 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.iscsi_initiator_list = [] {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.874796 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.use_multipath_io = False {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.874943 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.limit_cpu_features = False {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.875091 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_count = 10 {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.875223 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_count = 10 {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.875361 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_interval = 5 {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.875494 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_interval = 5 {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.875635 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.vswitch_name = None {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.875768 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_check_timeframe = 60 {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.875898 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.wait_soft_reboot_seconds = 60 {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.876028 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_event_polling_interval = 2 {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.876158 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] mks.enabled = False {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.876296 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.qemu_img_cmd = qemu-img.exe {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.876296 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.use_multipath_io = False {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.876574 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] mks.mksproxy_base_url = http://127.0.0.1:6090/ {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.876716 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_count = 10 {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.876855 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.manager_interval = 2400 {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.876994 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_interval = 5 {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.877134 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.precache_concurrency = 1 {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.877279 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.vswitch_name = None {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.877418 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_base_images = True {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.877557 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.wait_soft_reboot_seconds = 60 {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.877689 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_original_minimum_age_seconds = 86400 {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.877830 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] mks.enabled = False {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.877974 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_resized_minimum_age_seconds = 3600 {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.877974 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.subdirectory_name = _base {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.878222 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] mks.mksproxy_base_url = http://127.0.0.1:6090/ {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.878353 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_max_retries = 60 {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.878547 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.manager_interval = 2400 {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.878693 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_retry_interval = 2 {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.878834 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.precache_concurrency = 1 {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.878965 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_section = None {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.879095 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_base_images = True {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.879233 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_type = None {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.879909 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_original_minimum_age_seconds = 86400 {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.880053 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] ironic.cafile = None {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.880201 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_resized_minimum_age_seconds = 3600 {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.880343 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] ironic.certfile = None {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.880476 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.subdirectory_name = _base {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.880607 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] ironic.collect_timing = False {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.880746 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_max_retries = 60 {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.880886 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retries = None {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.881018 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_retry_interval = 2 {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.881167 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retry_delay = None {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.881299 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_section = None {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.881439 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] ironic.endpoint_override = None {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.881579 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_type = None {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.881711 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] ironic.insecure = False {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.881841 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] ironic.cafile = None {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.881987 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] ironic.keyfile = None {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.881987 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] ironic.max_version = None {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.882251 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] ironic.certfile = None {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.882389 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] ironic.min_version = None {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.882549 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] ironic.collect_timing = False {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.882682 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] ironic.partition_key = None {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.882822 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retries = None {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.882956 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] ironic.peer_list = [] {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.883088 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retry_delay = None {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.883226 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] ironic.region_name = None {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.883366 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] ironic.endpoint_override = None {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.883497 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] ironic.serial_console_state_timeout = 10 {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.883638 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] ironic.insecure = False {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.883777 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_name = None {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.883910 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] ironic.keyfile = None {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.884042 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_type = baremetal {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.884177 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] ironic.max_version = None {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.884316 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] ironic.split_loggers = False {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.884449 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] ironic.min_version = None {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.884579 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retries = None {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.884801 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] ironic.partition_key = None {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.884945 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retry_delay = None {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.885079 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] ironic.peer_list = [] {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.885225 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] ironic.timeout = None {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.885358 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] ironic.region_name = None {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.885497 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] ironic.valid_interfaces = ['internal', 'public'] {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.885619 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] ironic.serial_console_state_timeout = 10 {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.885756 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] ironic.version = None {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.885896 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_name = None {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.886040 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.backend = nova.keymgr.conf_key_mgr.ConfKeyManager {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.886175 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_type = baremetal {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.886312 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.fixed_key = **** {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.886451 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] ironic.split_loggers = False {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.886639 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] barbican.auth_endpoint = http://localhost/identity/v3 {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.886813 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retries = None {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.886955 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_api_version = None {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.887088 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retry_delay = None {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.887217 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint = None {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.887331 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] ironic.timeout = None {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.887441 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint_type = public {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.887562 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] ironic.valid_interfaces = ['internal', 'public'] {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.887684 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_region_name = None {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.887815 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] ironic.version = None {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.887946 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] barbican.cafile = None {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.888085 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.backend = nova.keymgr.conf_key_mgr.ConfKeyManager {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.888218 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] barbican.certfile = None {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.888362 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.fixed_key = **** {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.888501 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] barbican.collect_timing = False {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.888501 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] barbican.insecure = False {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.888759 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] barbican.auth_endpoint = http://localhost/identity/v3 {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.888898 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] barbican.keyfile = None {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.889044 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_api_version = None {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.889189 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] barbican.number_of_retries = 60 {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.889332 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint = None {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.889464 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] barbican.retry_delay = 1 {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.889595 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint_type = public {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.889732 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] barbican.send_service_user_token = False {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.889879 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_region_name = None {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.890018 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] barbican.split_loggers = False {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.890149 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] barbican.cafile = None {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.890287 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] barbican.timeout = None {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.890426 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] barbican.certfile = None {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.890596 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl = True {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.890740 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] barbican.collect_timing = False {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.890872 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl_path = None {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.891010 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] barbican.insecure = False {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.891142 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_section = None {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.891280 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] barbican.keyfile = None {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.891419 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_type = None {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.891561 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] barbican.number_of_retries = 60 {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.891692 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.cafile = None {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.891831 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] barbican.retry_delay = 1 {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.891962 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.certfile = None {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.892100 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] barbican.send_service_user_token = False {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.892232 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.collect_timing = False {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.892408 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] barbican.split_loggers = False {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.892550 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.insecure = False {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.892682 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] barbican.timeout = None {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.892801 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.keyfile = None {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.892922 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl = True {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.893062 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.split_loggers = False {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.893203 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl_path = None {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.893334 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.timeout = None {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.893474 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_section = None {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.893606 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_role_id = None {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.893736 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_type = None {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.893876 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_secret_id = None {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.894008 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.cafile = None {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.894158 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] vault.cafile = None {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.894290 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.certfile = None {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.894429 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] vault.certfile = None {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.894587 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.collect_timing = False {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.894729 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] vault.collect_timing = False {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.894971 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.insecure = False {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.895639 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] vault.insecure = False {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.895775 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.keyfile = None {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.895906 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] vault.keyfile = None {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.896081 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.split_loggers = False {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.896263 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_mountpoint = secret {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.896263 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_version = 2 {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.896516 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.timeout = None {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.896656 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] vault.namespace = None {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.896789 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] vault.root_token_id = None {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.896934 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_role_id = None {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.897067 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] vault.split_loggers = False {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.897199 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_secret_id = None {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.897329 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] vault.ssl_ca_crt_file = None {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.897469 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] vault.cafile = None {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.897609 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] vault.timeout = None {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.897756 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] vault.certfile = None {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.897897 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] vault.use_ssl = False {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.898030 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] vault.collect_timing = False {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.898204 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] vault.vault_url = http://127.0.0.1:8200 {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.898350 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] vault.insecure = False {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.898541 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] keystone.cafile = None {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.898697 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] vault.keyfile = None {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.898831 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] keystone.certfile = None {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.898970 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_mountpoint = secret {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.899138 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] keystone.collect_timing = False {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.899284 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_version = 2 {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.899447 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retries = None {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.899744 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] vault.namespace = None {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.899928 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retry_delay = None {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.900070 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] vault.root_token_id = None {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.900203 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] keystone.endpoint_override = None {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.900543 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] vault.split_loggers = False {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.900706 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] keystone.insecure = False {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.900952 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] vault.ssl_ca_crt_file = None {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.901086 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] keystone.keyfile = None {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.901217 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] vault.timeout = None {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.901347 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] keystone.max_version = None {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.901488 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] vault.use_ssl = False {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.901621 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] keystone.min_version = None {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.901769 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] vault.vault_url = http://127.0.0.1:8200 {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.902473 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] keystone.region_name = None {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.902655 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] keystone.cafile = None {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.902799 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_name = None {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.903102 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] keystone.certfile = None {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.903243 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_type = identity {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.903385 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] keystone.collect_timing = False {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.903519 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] keystone.split_loggers = False {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.903652 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retries = None {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.903785 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retries = None {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.903928 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retry_delay = None {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.904070 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retry_delay = None {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.904212 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] keystone.endpoint_override = None {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.904349 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] keystone.timeout = None {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.904481 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] keystone.insecure = False {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.904613 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] keystone.valid_interfaces = ['internal', 'public'] {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.904755 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] keystone.keyfile = None {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.904896 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] keystone.version = None {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.905127 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] keystone.max_version = None {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.905127 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] keystone.min_version = None {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.905379 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.connection_uri = {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.905519 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] keystone.region_name = None {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.905652 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_mode = None {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.905782 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_name = None {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.905912 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_model_extra_flags = [] {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.906044 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_type = identity {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.906173 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_models = [] {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.906316 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] keystone.split_loggers = False {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.906455 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_attempts = 8 {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.906648 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retries = None {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.906789 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_timeout = 20 {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.906930 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retry_delay = None {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.907063 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_cachemodes = [] {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.907208 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] keystone.timeout = None {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.907341 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_prefix = None {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.907341 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.enabled_perf_events = [] {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.907607 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] keystone.valid_interfaces = ['internal', 'public'] {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.907782 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.file_backed_memory = 0 {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.907916 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] keystone.version = None {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.908062 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.gid_maps = [] {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.908196 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.connection_uri = {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.908341 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_disk_discard = None {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.908482 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_mode = None {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.908623 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_machine_type = None {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.908755 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_model_extra_flags = [] {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.908896 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_ceph_conf = {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.909029 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_models = [] {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.909160 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_poll_interval = 15 {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.909291 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_attempts = 8 {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.909438 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_timeout = 600 {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.909651 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_timeout = 20 {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.909793 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_store_name = {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.909935 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_cachemodes = [] {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.910075 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_pool = rbd {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.910208 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_prefix = None {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.910368 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_type = default {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.910525 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.enabled_perf_events = [] {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.911187 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_volume_group = None {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.911374 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.file_backed_memory = 0 {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.911524 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_key = False {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.911701 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.gid_maps = [] {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.911872 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_partition = -2 {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.911872 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_password = False {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.912137 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_disk_discard = None {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.912275 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iscsi_iface = None {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.912409 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_machine_type = None {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.912539 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iser_use_multipath = False {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.912676 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_ceph_conf = {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.912817 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_bandwidth = 0 {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.912950 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_poll_interval = 15 {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.913081 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_completion_timeout = 800 {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.913222 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_timeout = 600 {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.913362 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime = 500 {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.913503 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_store_name = {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.913645 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_delay = 75 {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.913784 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_pool = rbd {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.913916 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_steps = 10 {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.914056 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_type = default {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.914201 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_inbound_addr = None {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.914348 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_volume_group = None {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.914499 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_auto_converge = False {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.914647 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_key = False {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.914781 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_post_copy = False {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.914914 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_partition = -2 {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.915090 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_scheme = None {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.915307 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_password = False {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.915445 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_timeout_action = abort {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.915578 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iscsi_iface = None {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.915724 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_tunnelled = False {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.915856 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iser_use_multipath = False {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.915987 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_uri = None {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.916129 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_bandwidth = 0 {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.916297 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_with_native_tls = False {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.916436 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_completion_timeout = 800 {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.916604 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.max_queues = None {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.916741 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime = 500 {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.916958 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.mem_stats_period_seconds = 10 {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.917094 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_delay = 75 {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.917233 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_options = None {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.917366 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_steps = 10 {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.917616 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_inbound_addr = None {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.917760 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.917935 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_auto_converge = False {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.918073 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_aoe_discover_tries = 3 {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.918250 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_iser_scan_tries = 5 {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.918393 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_post_copy = False {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.918587 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_scheme = None {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.918731 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_memory_encrypted_guests = None {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.918874 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_nvme_discover_tries = 5 {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.919006 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_timeout_action = abort {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.919190 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_pcie_ports = 0 {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.919327 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_tunnelled = False {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.919502 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_uri = None {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.919645 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_volume_scan_tries = 5 {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.919820 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.pmem_namespaces = [] {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.919964 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_with_native_tls = False {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.920127 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.max_queues = None {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.920262 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_client_cfg = None {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.920443 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.mem_stats_period_seconds = 10 {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.920590 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_mount_point_base = /opt/stack/data/nova/mnt {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.920758 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_options = None {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.920902 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_connect_timeout = 5 {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.921156 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retries = 12 {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.921299 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.921471 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retry_interval = 5 {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.921617 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_aoe_discover_tries = 3 {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.921781 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_secret_uuid = None {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.921925 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_iser_scan_tries = 5 {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.922089 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_user = None {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.922233 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_memory_encrypted_guests = None {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.922403 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.realtime_scheduler_priority = 1 {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.922574 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_nvme_discover_tries = 5 {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.922744 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_pcie_ports = 0 {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.922887 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.remote_filesystem_transport = ssh {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.923055 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_image_id = None {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.923190 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_volume_scan_tries = 5 {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.923363 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_kernel_id = None {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.923509 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.pmem_namespaces = [] {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.923669 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_ramdisk_id = None {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.923806 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_client_cfg = None {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.923997 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rng_dev_path = /dev/urandom {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.924142 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_mount_point_base = /opt/stack/data/nova/mnt {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.924288 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rx_queue_size = None {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.924430 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_connect_timeout = 5 {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.924618 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_options = {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.924759 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retries = 12 {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.925009 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retry_interval = 5 {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.925153 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.925310 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_secret_uuid = None {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.925510 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_compression = False {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.925654 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_user = None {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.925787 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_image_format = None {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.925919 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.realtime_scheduler_priority = 1 {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.926059 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshots_directory = /opt/stack/data/nova/instances/snapshots {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.926288 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.remote_filesystem_transport = ssh {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.927012 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sparse_logical_volumes = False {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.927354 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_enabled = False {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.927557 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_image_id = None {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.927694 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_group = tss {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.927879 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_kernel_id = None {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.928033 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_user = tss {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.928195 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_ramdisk_id = None {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.928332 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sysinfo_serial = unique {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.928531 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rng_dev_path = /dev/urandom {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.928717 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.tx_queue_size = None {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.928863 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rx_queue_size = None {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.929006 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.uid_maps = [] {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.929169 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_options = {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.929332 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.use_virtio_for_bridges = True {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.929539 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.virt_type = kvm {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.929683 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.929867 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear = zero {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.930012 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_compression = False {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.930182 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear_size = 0 {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.930319 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_image_format = None {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.930515 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_use_multipath = False {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.930675 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshots_directory = /opt/stack/data/nova/instances/snapshots {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.930818 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_cache_path = None {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.930960 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sparse_logical_volumes = False {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.931153 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_log_path = /var/log/vstorage/%(cluster_name)s/nova.log.gz {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.931293 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_enabled = False {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.931483 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_group = qemu {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.931630 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_group = tss {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.931802 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_opts = [] {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.931952 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_user = tss {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.932132 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_perms = 0770 {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.932278 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sysinfo_serial = unique {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.932507 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.tx_queue_size = None {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.932643 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_point_base = /opt/stack/data/nova/mnt {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.932830 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.uid_maps = [] {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.932967 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_user = stack {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.933149 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.use_virtio_for_bridges = True {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.933295 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.wait_soft_reboot_seconds = 120 {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.933479 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_section = None {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.933623 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.virt_type = kvm {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.933843 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_type = password {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.934043 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear = zero {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.934189 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] neutron.cafile = None {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.934380 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear_size = 0 {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.934532 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] neutron.certfile = None {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.934732 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_use_multipath = False {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.934870 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] neutron.collect_timing = False {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.935046 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_cache_path = None {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.935184 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retries = None {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.935376 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retry_delay = None {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.935528 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_log_path = /var/log/vstorage/%(cluster_name)s/nova.log.gz {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.935747 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_group = qemu {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.935889 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] neutron.default_floating_pool = public {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.936021 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] neutron.endpoint_override = None {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.936151 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_opts = [] {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.936295 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] neutron.extension_sync_interval = 600 {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.936471 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_perms = 0770 {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.936744 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] neutron.http_retries = 3 {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.936910 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_point_base = /opt/stack/data/nova/mnt {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.937093 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] neutron.insecure = False {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.937262 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_user = stack {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.937433 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.wait_soft_reboot_seconds = 120 {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.937577 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] neutron.keyfile = None {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.937747 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] neutron.max_version = None {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.937902 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_section = None {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.938085 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_type = password {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.938229 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] neutron.metadata_proxy_shared_secret = **** {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.938472 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] neutron.min_version = None {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.938666 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] neutron.cafile = None {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.938825 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] neutron.ovs_bridge = br-int {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.938976 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] neutron.certfile = None {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.939148 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] neutron.physnets = [] {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.939296 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] neutron.collect_timing = False {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.939478 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] neutron.region_name = RegionOne {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.939624 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retries = None {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.939805 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_metadata_proxy = True {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.939943 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retry_delay = None {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.940116 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_name = None {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.940340 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] neutron.default_floating_pool = public {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.940485 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] neutron.endpoint_override = None {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.940625 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_type = network {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.940806 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] neutron.extension_sync_interval = 600 {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.940951 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] neutron.split_loggers = False {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.941124 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] neutron.http_retries = 3 {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.941275 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retries = None {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.941416 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retry_delay = None {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.941556 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] neutron.insecure = False {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.941725 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] neutron.timeout = None {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.941885 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] neutron.keyfile = None {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.942220 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] neutron.valid_interfaces = ['internal', 'public'] {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.942563 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] neutron.version = None {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.942714 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] neutron.max_version = None {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.942890 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] notifications.bdms_in_notifications = False {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.943035 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] neutron.metadata_proxy_shared_secret = **** {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.943291 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] neutron.min_version = None {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.943445 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] notifications.default_level = INFO {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.943588 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notification_format = unversioned {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.943720 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] neutron.ovs_bridge = br-int {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.943862 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notify_on_state_change = None {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.944003 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] neutron.physnets = [] {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.944178 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] notifications.versioned_notifications_topics = ['versioned_notifications'] {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.944326 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] neutron.region_name = RegionOne {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.944509 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_metadata_proxy = True {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.944652 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] pci.alias = [] {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.944827 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_name = None {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.944965 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] pci.device_spec = [] {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.945138 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] pci.report_in_placement = False {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.945287 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_type = network {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.945456 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_section = None {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.945593 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] neutron.split_loggers = False {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.945749 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retries = None {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.945956 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_type = password {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.946101 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retry_delay = None {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.946243 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] placement.cafile = None {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.946376 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] neutron.timeout = None {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.946574 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] placement.certfile = None {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.946718 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] placement.collect_timing = False {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.946850 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] neutron.valid_interfaces = ['internal', 'public'] {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.946996 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retries = None {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.947131 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] neutron.version = None {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.947286 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retry_delay = None {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.947424 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] notifications.bdms_in_notifications = False {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.947594 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] placement.endpoint_override = None {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.947733 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] notifications.default_level = INFO {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.947905 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] placement.insecure = False {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.948052 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notification_format = unversioned {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.948213 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] placement.keyfile = None {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.948352 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notify_on_state_change = None {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.948519 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] placement.max_version = None {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.948665 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] notifications.versioned_notifications_topics = ['versioned_notifications'] {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.948812 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] placement.min_version = None {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.948954 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] pci.alias = [] {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.949136 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] placement.region_name = RegionOne {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.949274 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] pci.device_spec = [] {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.949443 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_name = None {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.949595 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] pci.report_in_placement = False {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.949768 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_type = placement {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.949905 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_section = None {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.950079 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] placement.split_loggers = False {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.950216 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_type = password {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.950385 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retries = None {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.950567 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] placement.cafile = None {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.950715 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retry_delay = None {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.950850 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] placement.certfile = None {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.951023 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] placement.timeout = None {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.951178 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] placement.collect_timing = False {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.951370 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] placement.valid_interfaces = ['internal', 'public'] {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.951521 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retries = None {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.951677 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] placement.version = None {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.951815 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retry_delay = None {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.951989 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] placement.endpoint_override = None {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.952124 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] quota.cores = 20 {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.952304 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] placement.insecure = False {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.952455 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] quota.count_usage_from_placement = False {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.952615 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] placement.keyfile = None {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.952760 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] quota.driver = nova.quota.DbQuotaDriver {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.952926 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] placement.max_version = None {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.953071 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_content_bytes = 10240 {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.953215 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] placement.min_version = None {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.953371 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_path_length = 255 {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.953571 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_files = 5 {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.953706 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] placement.region_name = RegionOne {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.953848 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] quota.instances = 10 {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.953989 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_name = None {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.954132 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] quota.key_pairs = 100 {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.954275 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_type = placement {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.954456 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] quota.metadata_items = 128 {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.954624 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] placement.split_loggers = False {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.954789 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retries = None {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.954926 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] quota.ram = 51200 {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.955097 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] quota.recheck_quota = True {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.955246 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retry_delay = None {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.955388 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] placement.timeout = None {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.955519 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_group_members = 10 {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.955720 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_groups = 10 {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.955857 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] placement.valid_interfaces = ['internal', 'public'] {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.956082 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] placement.version = None {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.956218 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] rdp.enabled = False {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.956366 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] quota.cores = 20 {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.956583 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] rdp.html5_proxy_base_url = http://127.0.0.1:6083/ {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.956723 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] quota.count_usage_from_placement = False {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.956913 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.discover_hosts_in_cells_interval = -1 {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.957052 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] quota.driver = nova.quota.DbQuotaDriver {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.957228 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.enable_isolated_aggregate_filtering = False {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.957374 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_content_bytes = 10240 {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.957588 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.image_metadata_prefilter = False {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.957868 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.limit_tenants_to_placement_aggregate = False {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.958067 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_path_length = 255 {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.958206 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_attempts = 3 {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.958421 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_files = 5 {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.958605 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_placement_results = 1000 {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.958783 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] quota.instances = 10 {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.958933 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.placement_aggregate_required_for_tenants = False {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.959109 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_availability_zone = True {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.959253 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] quota.key_pairs = 100 {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.959421 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_image_type_support = False {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.959567 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] quota.metadata_items = 128 {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.959732 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_routed_network_aggregates = False {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.959869 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] quota.ram = 51200 {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.960064 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.workers = 2 {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.960202 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] quota.recheck_quota = True {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.960388 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_namespace = None {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.960534 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_group_members = 10 {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.960715 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_groups = 10 {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.960869 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_separator = . {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.961047 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] rdp.enabled = False {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.961193 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.available_filters = ['nova.scheduler.filters.all_filters'] {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.961368 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.build_failure_weight_multiplier = 1000000.0 {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.961608 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] rdp.html5_proxy_base_url = http://127.0.0.1:6083/ {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.961747 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cpu_weight_multiplier = 1.0 {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.961947 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.discover_hosts_in_cells_interval = -1 {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.962096 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cross_cell_move_weight_multiplier = 1000000.0 {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.962272 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.enable_isolated_aggregate_filtering = False {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.962420 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.disk_weight_multiplier = 1.0 {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.962635 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.image_metadata_prefilter = False {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.962786 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.enabled_filters = ['AvailabilityZoneFilter', 'ComputeFilter', 'ComputeCapabilitiesFilter', 'ImagePropertiesFilter', 'ServerGroupAntiAffinityFilter', 'ServerGroupAffinityFilter', 'SameHostFilter', 'DifferentHostFilter'] {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.962962 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.limit_tenants_to_placement_aggregate = False {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.963101 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.host_subset_size = 1 {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.963269 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.image_properties_default_architecture = None {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.963406 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_attempts = 3 {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.963634 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.io_ops_weight_multiplier = -1.0 {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.963789 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_placement_results = 1000 {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.963924 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_hosts = [] {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.964057 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.placement_aggregate_required_for_tenants = False {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.964198 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_images = [] {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.964331 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_availability_zone = True {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.964501 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_instances_per_host = 50 {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.964648 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_image_type_support = False {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.964812 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_io_ops_per_host = 8 {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.964950 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_routed_network_aggregates = False {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.965128 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.pci_weight_multiplier = 1.0 {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.965266 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.workers = 2 {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.965444 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.ram_weight_multiplier = 1.0 {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.965582 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_namespace = None {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.965755 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.restrict_isolated_hosts_to_isolated_images = True {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.965902 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_separator = . {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.966052 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.shuffle_best_same_weighed_hosts = False {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.966261 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.available_filters = ['nova.scheduler.filters.all_filters'] {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.966415 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_affinity_weight_multiplier = 1.0 {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.966586 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.build_failure_weight_multiplier = 1000000.0 {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.966723 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_anti_affinity_weight_multiplier = 1.0 {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.966857 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cpu_weight_multiplier = 1.0 {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.967002 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.track_instance_changes = False {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.967137 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cross_cell_move_weight_multiplier = 1000000.0 {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.967336 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.weight_classes = ['nova.scheduler.weights.all_weighers'] {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.967495 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.disk_weight_multiplier = 1.0 {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.967657 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] metrics.required = True {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.967806 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.enabled_filters = ['AvailabilityZoneFilter', 'ComputeFilter', 'ComputeCapabilitiesFilter', 'ImagePropertiesFilter', 'ServerGroupAntiAffinityFilter', 'ServerGroupAffinityFilter', 'SameHostFilter', 'DifferentHostFilter'] {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.967974 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_multiplier = 1.0 {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.968120 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.host_subset_size = 1 {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.968289 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_of_unavailable = -10000.0 {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.968434 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.image_properties_default_architecture = None {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.968607 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_setting = [] {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.968746 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.io_ops_weight_multiplier = -1.0 {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.968926 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_hosts = [] {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.969131 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.base_url = ws://127.0.0.1:6083/ {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.969272 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_images = [] {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.969446 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.enabled = False {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.969584 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_instances_per_host = 50 {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.969782 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.port_range = 10000:20000 {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.969921 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_io_ops_per_host = 8 {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.970137 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.proxyclient_address = 127.0.0.1 {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.970279 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.pci_weight_multiplier = 1.0 {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.970413 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_host = 0.0.0.0 {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.970575 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.ram_weight_multiplier = 1.0 {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.971038 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_port = 6083 {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.971187 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.restrict_isolated_hosts_to_isolated_images = True {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.971358 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.shuffle_best_same_weighed_hosts = False {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.971494 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_section = None {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.971680 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_affinity_weight_multiplier = 1.0 {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.971826 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_type = password {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.972001 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_anti_affinity_weight_multiplier = 1.0 {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.972152 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] service_user.cafile = None {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.972332 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.track_instance_changes = False {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.972477 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] service_user.certfile = None {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.972669 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] service_user.collect_timing = False {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.972806 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.weight_classes = ['nova.scheduler.weights.all_weighers'] {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.972983 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] service_user.insecure = False {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.973658 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] metrics.required = True {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.974097 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] service_user.keyfile = None {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.974432 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] service_user.send_service_user_token = True {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.974777 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] service_user.split_loggers = False {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.974924 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_multiplier = 1.0 {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.975091 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] service_user.timeout = None {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.975280 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_of_unavailable = -10000.0 {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.975426 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] spice.agent_enabled = True {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.975627 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_setting = [] {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.975766 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] spice.enabled = False {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.976174 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.base_url = ws://127.0.0.1:6083/ {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.976400 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_base_url = http://127.0.0.1:6082/spice_auto.html {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.976539 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.enabled = False {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.976678 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_host = 0.0.0.0 {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.976846 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.port_range = 10000:20000 {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.976983 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_port = 6082 {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.977190 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.proxyclient_address = 127.0.0.1 {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.977341 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_listen = 127.0.0.1 {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.977524 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_proxyclient_address = 127.0.0.1 {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.977652 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_host = 0.0.0.0 {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.977845 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.baseapi = None {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.977991 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_port = 6083 {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.978158 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.cert = None {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.978303 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_section = None {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.978506 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.compute = auto {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.978645 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_type = password {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.978821 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.conductor = None {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.978961 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] service_user.cafile = None {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.979133 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.scheduler = None {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.979279 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] service_user.certfile = None {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.979450 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_section = None {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.979586 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] service_user.collect_timing = False {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.979763 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_type = None {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.979907 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] service_user.insecure = False {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.980053 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.cafile = None {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.980176 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] service_user.keyfile = None {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.980363 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.certfile = None {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.980508 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] service_user.send_service_user_token = True {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.980675 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.collect_timing = False {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.980811 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] service_user.split_loggers = False {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.980985 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.insecure = False {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.981132 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] service_user.timeout = None {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.981291 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.keyfile = None {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.981428 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] spice.agent_enabled = True {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.981608 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.split_loggers = False {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.981752 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] spice.enabled = False {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.981914 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.timeout = None {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.982209 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_base_url = http://127.0.0.1:6082/spice_auto.html {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.982353 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] vmware.api_retry_count = 10 {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.982571 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] vmware.ca_file = None {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.982707 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_host = 0.0.0.0 {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.982882 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cache_prefix = None {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.983027 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_port = 6082 {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.983188 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cluster_name = None {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.983332 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_listen = 127.0.0.1 {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.983501 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] vmware.connection_pool_size = 10 {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.983647 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_proxyclient_address = 127.0.0.1 {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.983792 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] vmware.console_delay_seconds = None {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.984001 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.baseapi = None {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.984123 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] vmware.datastore_regex = None {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.984264 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.cert = None {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.984397 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_ip = None {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.984529 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.compute = auto {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.984671 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_password = **** {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.984813 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.conductor = None {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.984978 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_port = 443 {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.985124 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.scheduler = None {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.985265 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_username = None {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.985399 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_section = None {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.985582 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] vmware.insecure = False {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.985734 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_type = None {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.985868 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] vmware.integration_bridge = None {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.986001 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.cafile = None {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.986181 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] vmware.maximum_objects = 100 {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.986327 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.certfile = None {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.986593 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_default_policy = None {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.986739 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.collect_timing = False {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.986883 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_enabled = False {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.987017 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.insecure = False {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.987149 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_wsdl_location = None {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.987292 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.keyfile = None {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.987456 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_log_dir = /opt/vmware/vspc {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.987602 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.split_loggers = False {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.987764 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_proxy_uri = None {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.987910 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.timeout = None {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.988056 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_service_uri = None {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.988203 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] vmware.api_retry_count = 10 {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.988375 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] vmware.task_poll_interval = 0.5 {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.988512 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] vmware.ca_file = None {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.988707 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] vmware.use_linked_clone = True {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.989399 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cache_prefix = None {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.989568 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_keymap = en-us {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.989884 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port = 5900 {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.990156 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cluster_name = None {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.990278 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port_total = 10000 {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.990504 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] vmware.connection_pool_size = 10 {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.990644 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] vnc.auth_schemes = ['none'] {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.990824 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] vmware.console_delay_seconds = None {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.990971 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] vnc.enabled = True {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.991141 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] vmware.datastore_regex = None {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.991417 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_base_url = http://127.0.0.1:6080/vnc_auto.html {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.991556 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_ip = None {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.991759 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_password = **** {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.991905 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_host = 0.0.0.0 {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.992079 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_port = 443 {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.992225 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_port = 6080 {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.992393 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_username = None {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.992531 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_listen = 127.0.0.1 {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.992711 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] vmware.insecure = False {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.992857 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_proxyclient_address = 127.0.0.1 {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.993024 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] vmware.integration_bridge = None {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.993170 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_ca_certs = None {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.993344 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] vmware.maximum_objects = 100 {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.993482 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_cert = None {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.993703 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_default_policy = None {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.993850 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_key = None {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.994068 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_enabled = False {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.994213 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_compute_service_check_for_ffu = False {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.994346 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_wsdl_location = None {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.994509 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_fallback_pcpu_query = False {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.994664 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_group_policy_check_upcall = False {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.994799 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_log_dir = /opt/vmware/vspc {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.994966 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_libvirt_livesnapshot = False {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.995104 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_proxy_uri = None {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.995282 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_rootwrap = False {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.995428 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_service_uri = None {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.995605 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] vmware.task_poll_interval = 0.5 {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.995741 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_numa_live_migration = False {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.995929 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] vmware.use_linked_clone = True {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.996071 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_qemu_monitor_announce_self = False {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.996247 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_keymap = en-us {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.996384 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.ensure_libvirt_rbd_instance_dir_cleanup = False {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.996572 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port = 5900 {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.996712 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.handle_virt_lifecycle_events = True {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.996892 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port_total = 10000 {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.997013 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.libvirt_disable_apic = False {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.997210 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.never_download_image_if_on_rbd = False {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.997331 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] vnc.auth_schemes = ['none'] {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.997529 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.reserve_disk_resource_for_image_cache = False {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.997634 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] vnc.enabled = True {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.997841 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_cpu_compare_on_dest = False {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.998096 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_base_url = http://127.0.0.1:6080/vnc_auto.html {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.998199 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_hypervisor_version_check_on_lm = False {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.998441 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_host = 0.0.0.0 {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.998560 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.unified_limits_count_pcpu_as_vcpu = False {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.998785 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.wait_for_vif_plugged_event_during_hard_reboot = [] {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.998891 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_port = 6080 {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.999123 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_listen = 127.0.0.1 {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.999223 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.api_paste_config = /etc/nova/api-paste.ini {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.999532 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.client_socket_timeout = 900 {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.999639 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_proxyclient_address = 127.0.0.1 {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.999767 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_ca_certs = None {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:44.999860 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.default_pool_size = 1000 {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:45.000083 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.keep_alive = True {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:45.000176 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_cert = None {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:45.000396 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_key = None {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:45.000490 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.max_header_line = 16384 {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:45.000711 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.secure_proxy_ssl_header = None {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:45.000814 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_compute_service_check_for_ffu = False {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:45.001023 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_ca_file = None {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:45.001126 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_fallback_pcpu_query = False {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:45.001339 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_cert_file = None {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:45.001433 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_group_policy_check_upcall = False {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:45.001655 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_libvirt_livesnapshot = False {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:45.001748 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_key_file = None {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:45.001972 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.tcp_keepidle = 600 {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:45.002144 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_rootwrap = False {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:45.002289 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_numa_live_migration = False {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:45.002386 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.wsgi_log_format = %(client_ip)s "%(request_line)s" status: %(status_code)s len: %(body_length)s time: %(wall_seconds).7f {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:45.002630 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_qemu_monitor_announce_self = False {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:45.002756 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] zvm.ca_file = None {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:45.002938 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] zvm.cloud_connector_url = None {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:45.003053 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.ensure_libvirt_rbd_instance_dir_cleanup = False {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:45.003255 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.handle_virt_lifecycle_events = True {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:45.003370 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] zvm.image_tmp_path = /opt/stack/data/nova/images {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:45.003573 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.libvirt_disable_apic = False {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:45.003688 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] zvm.reachable_timeout = 300 {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:45.003893 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.never_download_image_if_on_rbd = False {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:45.004010 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:45.004289 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.reserve_disk_resource_for_image_cache = False {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:45.004987 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:45.005318 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:45.005523 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_cpu_compare_on_dest = False {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:45.005633 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_process_name = {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:45.005856 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_hypervisor_version_check_on_lm = False {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:45.005957 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:45.006180 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.unified_limits_count_pcpu_as_vcpu = False {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:45.006280 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:45.006522 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.wait_for_vif_plugged_event_during_hard_reboot = [] {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:45.006640 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_new_defaults = False {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:45.006879 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.api_paste_config = /etc/nova/api-paste.ini {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:45.006975 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_scope = False {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:45.007210 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.client_socket_timeout = 900 {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:45.007312 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_default_rule = default {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:45.007538 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.default_pool_size = 1000 {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:45.007632 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:45.007864 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.keep_alive = True {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:45.007953 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_file = policy.yaml {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:45.008190 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.max_header_line = 16384 {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:45.008291 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:45.008510 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.secure_proxy_ssl_header = None {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:45.008604 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:45.008817 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:45.008909 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_ca_file = None {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:45.009126 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:45.009226 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_cert_file = None {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:45.009440 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:45.009541 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_key_file = None {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:45.009761 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_section = None {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:45.009907 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.tcp_keepidle = 600 {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:45.010075 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_type = None {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:45.010171 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.wsgi_log_format = %(client_ip)s "%(request_line)s" status: %(status_code)s len: %(body_length)s time: %(wall_seconds).7f {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:45.010384 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.cafile = None {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:45.010518 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] zvm.ca_file = None {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:45.010718 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.certfile = None {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:45.010897 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] zvm.cloud_connector_url = None {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:45.011062 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.collect_timing = False {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:45.011224 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] zvm.image_tmp_path = /opt/stack/data/nova/images {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:45.011374 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retries = None {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:45.011523 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] zvm.reachable_timeout = 300 {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:45.011678 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retry_delay = None {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:45.011833 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:45.011993 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_id = None {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:45.012156 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:45.012305 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_override = None {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:45.012460 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:45.012614 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.insecure = False {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:45.012761 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_process_name = {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:45.012936 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.keyfile = None {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:45.013099 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:45.013248 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.max_version = None {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:45.013395 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.min_version = None {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:45.013540 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:45.013687 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_new_defaults = False {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:45.013845 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.region_name = None {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:45.014003 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_scope = False {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:45.014178 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_name = None {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:45.014435 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_default_rule = default {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:45.014625 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_type = None {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:45.014786 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:45.014949 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.split_loggers = False {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:45.015108 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retries = None {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:45.015255 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_file = policy.yaml {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:45.015402 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retry_delay = None {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:45.015558 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:45.015705 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.timeout = None {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:45.015863 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:45.016021 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.valid_interfaces = None {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:45.016170 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:45.016318 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.version = None {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:45.016464 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:45.016625 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:45.016794 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:45.017020 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] profiler.connection_string = messaging:// {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:45.017182 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_section = None {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:45.017332 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] profiler.enabled = False {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:45.017479 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_type = None {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:45.017479 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.cafile = None {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:45.017773 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_doc_type = notification {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:45.017929 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.certfile = None {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:45.018124 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_size = 10000 {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:45.018284 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.collect_timing = False {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:45.018441 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_time = 2m {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:45.018646 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retries = None {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:45.018804 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] profiler.filter_error_trace = False {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:45.018960 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retry_delay = None {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:45.019108 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] profiler.hmac_keys = SECRET_KEY {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:45.019257 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_id = None {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:45.019411 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] profiler.sentinel_service_name = mymaster {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:45.019575 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_override = None {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:45.019726 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] profiler.socket_timeout = 0.1 {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:45.019873 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.insecure = False {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:45.020076 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] profiler.trace_sqlalchemy = False {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:45.020752 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.keyfile = None {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:45.020911 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:45.020911 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:45.021198 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:45.021345 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.max_version = None {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:45.021499 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:45.021657 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.min_version = None {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:45.021814 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:45.021963 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.region_name = None {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:45.022117 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:45.022272 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_name = None {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:45.022434 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_in_pthread = False {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:45.022611 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_type = None {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:45.022724 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_rate = 3 {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:45.022821 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.split_loggers = False {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:45.022980 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_timeout_threshold = 600 {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:45.023092 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retries = None {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:45.023295 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_compression = None {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:45.023413 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retry_delay = None {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:45.023623 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:45.023721 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.timeout = None {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:45.023935 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.valid_interfaces = None {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:45.024030 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:45.024248 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.version = None {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:45.024351 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:45.024631 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:45.024738 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:45.024874 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:45.024970 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] profiler.connection_string = messaging:// {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:45.025209 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:45.025317 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] profiler.enabled = False {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:45.025523 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:45.025625 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_doc_type = notification {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:45.025838 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_delivery_limit = 0 {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:45.025939 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_size = 10000 {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:45.026151 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_queue = False {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:45.026254 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_time = 2m {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:45.026464 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_bytes = 0 {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:45.026626 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] profiler.filter_error_trace = False {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:45.026798 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_length = 0 {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:45.026903 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] profiler.hmac_keys = SECRET_KEY {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:45.027118 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:45.027212 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] profiler.sentinel_service_name = mymaster {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:45.027431 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:45.027532 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] profiler.socket_timeout = 0.1 {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:45.027743 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:45.027836 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] profiler.trace_sqlalchemy = False {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:45.028063 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:45.028178 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:45.028380 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl = False {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:45.028474 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:45.028697 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:45.028800 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_ca_file = {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:45.029019 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:45.029113 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_cert_file = {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:45.029329 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_enforce_fips_mode = False {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:45.029421 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:45.029641 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:45.029741 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_key_file = {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:45.029963 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_in_pthread = False {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:45.030065 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_version = {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:45.030296 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_rate = 3 {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:45.030389 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.driver = ['messagingv2'] {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:45.030635 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.retry = -1 {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:45.030747 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_timeout_threshold = 600 {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:45.030948 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_compression = None {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:45.031040 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.topics = ['notifications'] {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:45.031280 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:45.031395 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.transport_url = **** {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:45.031604 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler = None {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:45.031710 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:45.031924 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler_interval = 1 {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:45.032026 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:45.032230 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.log_dir = None {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:45.032324 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:45.032475 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=125640) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Oct 19 00:39:45.032645 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:45.032871 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:45.033188 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:45.033509 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_delivery_limit = 0 {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:45.033825 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_queue = False {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:45.034140 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_bytes = 0 {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:45.034457 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_length = 0 {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:45.034801 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:45.035115 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:45.035431 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:45.036175 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:45.036508 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl = False {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:45.036848 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_ca_file = {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:45.037183 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_cert_file = {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:45.037503 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_enforce_fips_mode = False {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:45.037839 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_key_file = {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:45.038175 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_version = {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:45.038548 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.driver = ['messagingv2'] {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:45.038871 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.retry = -1 {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:45.039223 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.topics = ['notifications'] {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:45.039559 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.transport_url = **** {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:45.039887 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler = None {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:45.040208 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler_interval = 1 {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:45.040521 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.log_dir = None {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 19 00:39:45.040770 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=125641) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Oct 19 00:39:45.104421 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: INFO dbcounter [None req-2ab87d32-9740-4272-bc59-f5b8086695b2 None None] Registered counter for database nova_api Oct 19 00:39:45.106225 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO dbcounter [None req-ce7d6826-e489-49d1-aaeb-1514e98f673a None None] Registered counter for database nova_api Oct 19 00:39:45.126044 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_db.sqlalchemy.engines [None req-2ab87d32-9740-4272-bc59-f5b8086695b2 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=125640) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 19 00:39:45.127609 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_db.sqlalchemy.engines [None req-ce7d6826-e489-49d1-aaeb-1514e98f673a None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=125641) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 19 00:39:45.133382 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG dbcounter [-] [125640] Writer thread running {{(pid=125640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 19 00:39:45.134988 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG dbcounter [-] [125641] Writer thread running {{(pid=125641) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 19 00:39:45.194086 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.context [None req-ce7d6826-e489-49d1-aaeb-1514e98f673a None None] Found 2 cells: 00000000-0000-0000-0000-000000000000(cell0),3d98fd57-b040-4364-b7a0-5303a14589e8(cell1) {{(pid=125641) load_cells /opt/stack/nova/nova/context.py:464}} Oct 19 00:39:45.195095 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.context [None req-2ab87d32-9740-4272-bc59-f5b8086695b2 None None] Found 2 cells: 00000000-0000-0000-0000-000000000000(cell0),3d98fd57-b040-4364-b7a0-5303a14589e8(cell1) {{(pid=125640) load_cells /opt/stack/nova/nova/context.py:464}} Oct 19 00:39:45.198167 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ce7d6826-e489-49d1-aaeb-1514e98f673a None None] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:39:45.198674 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ce7d6826-e489-49d1-aaeb-1514e98f673a None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:39:45.199250 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-2ab87d32-9740-4272-bc59-f5b8086695b2 None None] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:39:45.199732 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-2ab87d32-9740-4272-bc59-f5b8086695b2 None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:39:45.200279 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ce7d6826-e489-49d1-aaeb-1514e98f673a None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:39:45.201007 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ce7d6826-e489-49d1-aaeb-1514e98f673a None None] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:39:45.201387 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-2ab87d32-9740-4272-bc59-f5b8086695b2 None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:39:45.201508 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ce7d6826-e489-49d1-aaeb-1514e98f673a None None] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:39:45.202133 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-2ab87d32-9740-4272-bc59-f5b8086695b2 None None] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:39:45.202688 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-2ab87d32-9740-4272-bc59-f5b8086695b2 None None] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:39:45.203501 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ce7d6826-e489-49d1-aaeb-1514e98f673a None None] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:39:45.204784 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-2ab87d32-9740-4272-bc59-f5b8086695b2 None None] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:39:45.205483 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO dbcounter [None req-ce7d6826-e489-49d1-aaeb-1514e98f673a None None] Registered counter for database nova_cell0 Oct 19 00:39:45.206915 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: INFO dbcounter [None req-2ab87d32-9740-4272-bc59-f5b8086695b2 None None] Registered counter for database nova_cell0 Oct 19 00:39:45.210696 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO dbcounter [None req-ce7d6826-e489-49d1-aaeb-1514e98f673a None None] Registered counter for database nova_cell1 Oct 19 00:39:45.211805 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: INFO dbcounter [None req-2ab87d32-9740-4272-bc59-f5b8086695b2 None None] Registered counter for database nova_cell1 Oct 19 00:39:45.218573 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_db.sqlalchemy.engines [None req-ce7d6826-e489-49d1-aaeb-1514e98f673a None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=125641) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 19 00:39:45.220247 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_db.sqlalchemy.engines [None req-2ab87d32-9740-4272-bc59-f5b8086695b2 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=125640) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 19 00:39:45.222853 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_db.sqlalchemy.engines [None req-ce7d6826-e489-49d1-aaeb-1514e98f673a None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=125641) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 19 00:39:45.225124 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_db.sqlalchemy.engines [None req-2ab87d32-9740-4272-bc59-f5b8086695b2 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=125640) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 19 00:39:45.227771 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG dbcounter [-] [125641] Writer thread running {{(pid=125641) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 19 00:39:45.231516 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG dbcounter [-] [125640] Writer thread running {{(pid=125640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 19 00:39:45.232933 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG dbcounter [-] [125640] Writer thread running {{(pid=125640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 19 00:39:45.376076 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG dbcounter [-] [125641] Writer thread running {{(pid=125641) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 19 00:39:45.383023 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: INFO dbcounter [None req-2ab87d32-9740-4272-bc59-f5b8086695b2 None None] Registered counter for database nova_cell0 Oct 19 00:39:45.385760 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO dbcounter [None req-ce7d6826-e489-49d1-aaeb-1514e98f673a None None] Registered counter for database nova_cell0 Oct 19 00:39:45.391473 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_db.sqlalchemy.engines [None req-2ab87d32-9740-4272-bc59-f5b8086695b2 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=125640) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 19 00:39:45.394624 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_db.sqlalchemy.engines [None req-ce7d6826-e489-49d1-aaeb-1514e98f673a None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=125641) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 19 00:39:45.398144 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG dbcounter [-] [125640] Writer thread running {{(pid=125640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 19 00:39:45.400390 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG dbcounter [-] [125641] Writer thread running {{(pid=125641) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 19 00:39:45.721255 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: WARNING keystonemiddleware.auth_token [None req-ce7d6826-e489-49d1-aaeb-1514e98f673a None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Oct 19 00:39:45.727126 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: WARNING keystonemiddleware.auth_token [None req-2ab87d32-9740-4272-bc59-f5b8086695b2 None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Oct 19 00:39:45.729406 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: WARNING keystonemiddleware.auth_token [None req-ce7d6826-e489-49d1-aaeb-1514e98f673a None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Oct 19 00:39:45.734440 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: WARNING keystonemiddleware.auth_token [None req-2ab87d32-9740-4272-bc59-f5b8086695b2 None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Oct 19 00:39:46.078847 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: WARNING keystonemiddleware.auth_token [None req-ce7d6826-e489-49d1-aaeb-1514e98f673a None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Oct 19 00:39:46.085634 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: WARNING keystonemiddleware.auth_token [None req-ce7d6826-e489-49d1-aaeb-1514e98f673a None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Oct 19 00:39:46.090439 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: WARNING keystonemiddleware.auth_token [None req-2ab87d32-9740-4272-bc59-f5b8086695b2 None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Oct 19 00:39:46.090923 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x5615d8e729c0 pid: 125641 (default app) Oct 19 00:39:46.097317 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: WARNING keystonemiddleware.auth_token [None req-2ab87d32-9740-4272-bc59-f5b8086695b2 None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Oct 19 00:39:46.103451 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x5615d8e729c0 pid: 125640 (default app) Oct 19 00:40:21.679488 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG dbcounter [-] [125641] Writing DB stats nova_api:SELECT=3 {{(pid=125641) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 00:40:21.680406 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG dbcounter [-] [125641] Writing DB stats nova_cell1:SELECT=3 {{(pid=125641) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 00:40:21.681183 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG dbcounter [-] [125641] Writing DB stats nova_cell0:SELECT=3 {{(pid=125641) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 00:40:21.681949 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG dbcounter [-] [125641] Writing DB stats nova_cell0:SELECT=3 {{(pid=125641) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 00:40:22.817201 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-4e809463-25c8-4042-b50e-6e1af3f59f9c tempest-VolumesNegativeTest-1777702828 tempest-VolumesNegativeTest-1777702828-project] Action: 'create', calling method: .version_select at 0x7f71c8e45b80>, body: {"volume": {"size": "#$%", "display_name": "tempest-VolumesNegativeTest-Volume-1513990897", "metadata": {"Type": "work"}}} {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 19 00:40:22.824214 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-4e809463-25c8-4042-b50e-6e1af3f59f9c tempest-VolumesNegativeTest-1777702828 tempest-VolumesNegativeTest-1777702828-project] Request method failure captured: Oct 19 00:40:22.824214 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: request: POST /compute/v2.1/os-volumes HTTP/1.1 Oct 19 00:40:22.824214 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Accept: application/json Oct 19 00:40:22.824214 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Accept-Encoding: identity Oct 19 00:40:22.824214 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Connection: close Oct 19 00:40:22.824214 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Content-Length: 122 Oct 19 00:40:22.824214 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Content-Type: application/json Oct 19 00:40:22.824214 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Host: 10.222.0.26 Oct 19 00:40:22.824214 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Openstack-System-Scope: None Oct 19 00:40:22.824214 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: User-Agent: python-urllib3/1.26.12 Oct 19 00:40:22.824214 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Auth-Token: gAAAAABjT0d1daPsFvMJjSOTOV_G5OpgqDipoTmAzhu8K9YedqGdEfTFjeo5Tm_5A0vyaGOXa_JjRufNWIm5eeq3jVi34mVfYCxOeql1o4l3a11zoMxtDaoeiUXW9sPbOWgHA2J_SPeKzdHcB181eO-WC8M7UG0lX3mF62HWBLnwXvR9Npf1HTQ Oct 19 00:40:22.824214 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Domain-Id: None Oct 19 00:40:22.824214 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Domain-Name: None Oct 19 00:40:22.824214 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Identity-Status: Confirmed Oct 19 00:40:22.824214 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Is-Admin-Project: True Oct 19 00:40:22.824214 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Project-Domain-Id: default Oct 19 00:40:22.824214 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Project-Domain-Name: Default Oct 19 00:40:22.824214 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Project-Id: 9e9333ca01214933af63450c15a2b51b Oct 19 00:40:22.824214 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Project-Name: tempest-VolumesNegativeTest-1777702828 Oct 19 00:40:22.824214 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Role: member,reader Oct 19 00:40:22.824214 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Roles: member,reader Oct 19 00:40:22.826101 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/9e9333ca01214933af63450c15a2b51b"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/9e9333ca01214933af63450c15a2b51b"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/9e9333ca01214933af63450c15a2b51b"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_9e9333ca01214933af63450c15a2b51b"}]}] Oct 19 00:40:22.826101 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Tenant: tempest-VolumesNegativeTest-1777702828 Oct 19 00:40:22.826101 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Tenant-Id: 9e9333ca01214933af63450c15a2b51b Oct 19 00:40:22.826101 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Tenant-Name: tempest-VolumesNegativeTest-1777702828 Oct 19 00:40:22.826101 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User: tempest-VolumesNegativeTest-1777702828-project Oct 19 00:40:22.826101 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User-Domain-Id: default Oct 19 00:40:22.826101 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User-Domain-Name: Default Oct 19 00:40:22.826101 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User-Id: e9740ef47b0145c18430a28c0e53a0d6 Oct 19 00:40:22.826101 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User-Name: tempest-VolumesNegativeTest-1777702828-project Oct 19 00:40:22.826101 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Oct 19 00:40:22.826101 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: {"volume": {"size": "#$%", "display_name": "tempest-VolumesNegativeTest-Volume-1513990897", "metadata": {"Type": "work"}}} Oct 19 00:40:22.826101 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: method: Oct 19 00:40:22.826101 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: exception: Invalid input for field/attribute size. Value: #$%. '#$%' does not match '^[0-9]+$' Oct 19 00:40:22.827036 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: action_args: {'body': {'volume': {'size': '#$%', 'display_name': 'tempest-VolumesNegativeTest-Volume-1513990897', 'metadata': {'Type': 'work'}}}} Oct 19 00:40:22.827036 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 19 00:40:22.827036 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 19 00:40:22.827036 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 302, in validate Oct 19 00:40:22.827036 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) Oct 19 00:40:22.827036 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 303, in validate Oct 19 00:40:22.827036 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi raise error Oct 19 00:40:22.827036 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: '#$%' does not match '^[0-9]+$' Oct 19 00:40:22.827036 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:40:22.827036 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Failed validating 'pattern' in schema['properties']['volume']['properties']['size']: Oct 19 00:40:22.827036 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi {'minimum': 1, 'pattern': '^[0-9]+$', 'type': ['integer', 'string']} Oct 19 00:40:22.827036 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:40:22.827036 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi On instance['volume']: Oct 19 00:40:22.827036 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi '#$%' Oct 19 00:40:22.827036 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:40:22.827036 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 19 00:40:22.827036 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:40:22.827036 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 19 00:40:22.827036 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 19 00:40:22.827036 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 19 00:40:22.827036 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 19 00:40:22.827036 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 19 00:40:22.828399 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 19 00:40:22.828399 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 19 00:40:22.828399 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 19 00:40:22.828399 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 19 00:40:22.828399 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper Oct 19 00:40:22.828399 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], Oct 19 00:40:22.828399 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper Oct 19 00:40:22.828399 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) Oct 19 00:40:22.828399 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 331, in validate Oct 19 00:40:22.828399 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) Oct 19 00:40:22.828399 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute size. Value: #$%. '#$%' does not match '^[0-9]+$' Oct 19 00:40:22.828399 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:40:22.828399 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 19 00:40:22.828399 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:40:22.828399 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 19 00:40:22.828399 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 19 00:40:22.828399 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 19 00:40:22.828399 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ Oct 19 00:40:22.828399 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( Oct 19 00:40:22.829557 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute size. Value: #$%. '#$%' does not match '^[0-9]+$' Oct 19 00:40:22.829557 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:40:22.829557 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-4e809463-25c8-4042-b50e-6e1af3f59f9c tempest-VolumesNegativeTest-1777702828 tempest-VolumesNegativeTest-1777702828-project] Returning 400 to user: Invalid input for field/attribute size. Value: #$%. '#$%' does not match '^[0-9]+$' {{(pid=125641) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 19 00:40:22.829557 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-4e809463-25c8-4042-b50e-6e1af3f59f9c tempest-VolumesNegativeTest-1777702828 tempest-VolumesNegativeTest-1777702828-project] 10.222.0.26 "POST /compute/v2.1/os-volumes" status: 400 len: 127 microversion: 2.1 time: 1.157163 Oct 19 00:40:22.829557 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 1/1] 10.222.0.26 () {60 vars in 1156 bytes} [Wed Oct 19 00:40:21 2022] POST /compute/v2.1/os-volumes => generated 127 bytes in 1159 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Oct 19 00:40:23.069578 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi [None req-979eecd7-5d7c-4228-b3ee-31b0e7432f28 tempest-VolumesNegativeTest-1777702828 tempest-VolumesNegativeTest-1777702828-project] Action: 'create', calling method: .version_select at 0x7f71c8ef7670>, body: {"volume": {"size": "0", "display_name": "tempest-VolumesNegativeTest-Volume-1918167065", "metadata": {"Type": "work"}}} {{(pid=125640) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 19 00:40:23.076442 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi [None req-979eecd7-5d7c-4228-b3ee-31b0e7432f28 tempest-VolumesNegativeTest-1777702828 tempest-VolumesNegativeTest-1777702828-project] Request method failure captured: Oct 19 00:40:23.076442 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: request: POST /compute/v2.1/os-volumes HTTP/1.1 Oct 19 00:40:23.076442 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: Accept: application/json Oct 19 00:40:23.076442 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: Accept-Encoding: identity Oct 19 00:40:23.076442 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: Connection: close Oct 19 00:40:23.076442 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: Content-Length: 120 Oct 19 00:40:23.076442 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: Content-Type: application/json Oct 19 00:40:23.076442 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: Host: 10.222.0.26 Oct 19 00:40:23.076442 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: Openstack-System-Scope: None Oct 19 00:40:23.076442 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: User-Agent: python-urllib3/1.26.12 Oct 19 00:40:23.076442 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: X-Auth-Token: gAAAAABjT0d1daPsFvMJjSOTOV_G5OpgqDipoTmAzhu8K9YedqGdEfTFjeo5Tm_5A0vyaGOXa_JjRufNWIm5eeq3jVi34mVfYCxOeql1o4l3a11zoMxtDaoeiUXW9sPbOWgHA2J_SPeKzdHcB181eO-WC8M7UG0lX3mF62HWBLnwXvR9Npf1HTQ Oct 19 00:40:23.076442 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: X-Domain-Id: None Oct 19 00:40:23.076442 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: X-Domain-Name: None Oct 19 00:40:23.076442 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: X-Identity-Status: Confirmed Oct 19 00:40:23.076442 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: X-Is-Admin-Project: True Oct 19 00:40:23.076442 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: X-Project-Domain-Id: default Oct 19 00:40:23.076442 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: X-Project-Domain-Name: Default Oct 19 00:40:23.076442 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: X-Project-Id: 9e9333ca01214933af63450c15a2b51b Oct 19 00:40:23.076442 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: X-Project-Name: tempest-VolumesNegativeTest-1777702828 Oct 19 00:40:23.076442 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: X-Role: member,reader Oct 19 00:40:23.076442 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: X-Roles: member,reader Oct 19 00:40:23.077978 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/9e9333ca01214933af63450c15a2b51b"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/9e9333ca01214933af63450c15a2b51b"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/9e9333ca01214933af63450c15a2b51b"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_9e9333ca01214933af63450c15a2b51b"}]}] Oct 19 00:40:23.077978 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: X-Tenant: tempest-VolumesNegativeTest-1777702828 Oct 19 00:40:23.077978 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: X-Tenant-Id: 9e9333ca01214933af63450c15a2b51b Oct 19 00:40:23.077978 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: X-Tenant-Name: tempest-VolumesNegativeTest-1777702828 Oct 19 00:40:23.077978 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: X-User: tempest-VolumesNegativeTest-1777702828-project Oct 19 00:40:23.077978 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: X-User-Domain-Id: default Oct 19 00:40:23.077978 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: X-User-Domain-Name: Default Oct 19 00:40:23.077978 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: X-User-Id: e9740ef47b0145c18430a28c0e53a0d6 Oct 19 00:40:23.077978 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: X-User-Name: tempest-VolumesNegativeTest-1777702828-project Oct 19 00:40:23.077978 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: Oct 19 00:40:23.077978 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: {"volume": {"size": "0", "display_name": "tempest-VolumesNegativeTest-Volume-1918167065", "metadata": {"Type": "work"}}} Oct 19 00:40:23.077978 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: method: Oct 19 00:40:23.077978 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: exception: Invalid input for field/attribute size. Value: 0. 0 is less than the minimum of 1 Oct 19 00:40:23.079065 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: action_args: {'body': {'volume': {'size': '0', 'display_name': 'tempest-VolumesNegativeTest-Volume-1918167065', 'metadata': {'Type': 'work'}}}} Oct 19 00:40:23.079065 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: {{(pid=125640) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 19 00:40:23.079065 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 19 00:40:23.079065 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 302, in validate Oct 19 00:40:23.079065 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) Oct 19 00:40:23.079065 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 303, in validate Oct 19 00:40:23.079065 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi raise error Oct 19 00:40:23.079065 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: 0 is less than the minimum of 1 Oct 19 00:40:23.079065 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi Oct 19 00:40:23.079065 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi Failed validating 'minimum' in schema['properties']['volume']['properties']['size']: Oct 19 00:40:23.079065 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi {'minimum': 1, 'pattern': '^[0-9]+$', 'type': ['integer', 'string']} Oct 19 00:40:23.079065 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi Oct 19 00:40:23.079065 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi On instance['volume']: Oct 19 00:40:23.079065 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi '0' Oct 19 00:40:23.079065 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi Oct 19 00:40:23.079065 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 19 00:40:23.079065 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi Oct 19 00:40:23.079065 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 19 00:40:23.079065 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 19 00:40:23.079065 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 19 00:40:23.079065 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 19 00:40:23.079065 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 19 00:40:23.080629 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 19 00:40:23.080629 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 19 00:40:23.080629 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 19 00:40:23.080629 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 19 00:40:23.080629 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper Oct 19 00:40:23.080629 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], Oct 19 00:40:23.080629 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper Oct 19 00:40:23.080629 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) Oct 19 00:40:23.080629 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 331, in validate Oct 19 00:40:23.080629 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) Oct 19 00:40:23.080629 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute size. Value: 0. 0 is less than the minimum of 1 Oct 19 00:40:23.080629 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi Oct 19 00:40:23.080629 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 19 00:40:23.080629 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi Oct 19 00:40:23.080629 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 19 00:40:23.080629 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 19 00:40:23.080629 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 19 00:40:23.080629 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ Oct 19 00:40:23.080629 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( Oct 19 00:40:23.082152 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute size. Value: 0. 0 is less than the minimum of 1 Oct 19 00:40:23.082152 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi Oct 19 00:40:23.083107 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi [None req-979eecd7-5d7c-4228-b3ee-31b0e7432f28 tempest-VolumesNegativeTest-1777702828 tempest-VolumesNegativeTest-1777702828-project] Returning 400 to user: Invalid input for field/attribute size. Value: 0. 0 is less than the minimum of 1 {{(pid=125640) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 19 00:40:23.084362 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: INFO nova.api.openstack.requestlog [None req-979eecd7-5d7c-4228-b3ee-31b0e7432f28 tempest-VolumesNegativeTest-1777702828 tempest-VolumesNegativeTest-1777702828-project] 10.222.0.26 "POST /compute/v2.1/os-volumes" status: 400 len: 125 microversion: 2.1 time: 0.251007 Oct 19 00:40:23.085411 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: [pid: 125640|app: 0|req: 1/2] 10.222.0.26 () {60 vars in 1156 bytes} [Wed Oct 19 00:40:22 2022] POST /compute/v2.1/os-volumes => generated 125 bytes in 254 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Oct 19 00:40:23.096605 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-95ffc4ba-76be-43c1-9bd2-48142d94f41c tempest-VolumesNegativeTest-1777702828 tempest-VolumesNegativeTest-1777702828-project] Action: 'create', calling method: .version_select at 0x7f71c8e63430>, body: {"volume": {"size": "", "display_name": "tempest-VolumesNegativeTest-Volume-656736892", "metadata": {"Type": "work"}}} {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 19 00:40:23.103917 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-95ffc4ba-76be-43c1-9bd2-48142d94f41c tempest-VolumesNegativeTest-1777702828 tempest-VolumesNegativeTest-1777702828-project] Request method failure captured: Oct 19 00:40:23.103917 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: request: POST /compute/v2.1/os-volumes HTTP/1.1 Oct 19 00:40:23.103917 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Accept: application/json Oct 19 00:40:23.103917 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Accept-Encoding: identity Oct 19 00:40:23.103917 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Connection: close Oct 19 00:40:23.103917 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Content-Length: 118 Oct 19 00:40:23.103917 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Content-Type: application/json Oct 19 00:40:23.103917 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Host: 10.222.0.26 Oct 19 00:40:23.103917 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Openstack-System-Scope: None Oct 19 00:40:23.103917 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: User-Agent: python-urllib3/1.26.12 Oct 19 00:40:23.103917 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Auth-Token: gAAAAABjT0d1daPsFvMJjSOTOV_G5OpgqDipoTmAzhu8K9YedqGdEfTFjeo5Tm_5A0vyaGOXa_JjRufNWIm5eeq3jVi34mVfYCxOeql1o4l3a11zoMxtDaoeiUXW9sPbOWgHA2J_SPeKzdHcB181eO-WC8M7UG0lX3mF62HWBLnwXvR9Npf1HTQ Oct 19 00:40:23.103917 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Domain-Id: None Oct 19 00:40:23.103917 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Domain-Name: None Oct 19 00:40:23.103917 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Identity-Status: Confirmed Oct 19 00:40:23.103917 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Is-Admin-Project: True Oct 19 00:40:23.103917 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Project-Domain-Id: default Oct 19 00:40:23.103917 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Project-Domain-Name: Default Oct 19 00:40:23.103917 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Project-Id: 9e9333ca01214933af63450c15a2b51b Oct 19 00:40:23.103917 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Project-Name: tempest-VolumesNegativeTest-1777702828 Oct 19 00:40:23.103917 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Role: member,reader Oct 19 00:40:23.103917 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Roles: member,reader Oct 19 00:40:23.105472 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/9e9333ca01214933af63450c15a2b51b"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/9e9333ca01214933af63450c15a2b51b"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/9e9333ca01214933af63450c15a2b51b"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_9e9333ca01214933af63450c15a2b51b"}]}] Oct 19 00:40:23.105472 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Tenant: tempest-VolumesNegativeTest-1777702828 Oct 19 00:40:23.105472 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Tenant-Id: 9e9333ca01214933af63450c15a2b51b Oct 19 00:40:23.105472 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Tenant-Name: tempest-VolumesNegativeTest-1777702828 Oct 19 00:40:23.105472 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User: tempest-VolumesNegativeTest-1777702828-project Oct 19 00:40:23.105472 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User-Domain-Id: default Oct 19 00:40:23.105472 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User-Domain-Name: Default Oct 19 00:40:23.105472 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User-Id: e9740ef47b0145c18430a28c0e53a0d6 Oct 19 00:40:23.105472 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User-Name: tempest-VolumesNegativeTest-1777702828-project Oct 19 00:40:23.105472 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Oct 19 00:40:23.105472 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: {"volume": {"size": "", "display_name": "tempest-VolumesNegativeTest-Volume-656736892", "metadata": {"Type": "work"}}} Oct 19 00:40:23.105472 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: method: Oct 19 00:40:23.105472 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: exception: Invalid input for field/attribute size. Value: . '' does not match '^[0-9]+$' Oct 19 00:40:23.106414 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: action_args: {'body': {'volume': {'size': '', 'display_name': 'tempest-VolumesNegativeTest-Volume-656736892', 'metadata': {'Type': 'work'}}}} Oct 19 00:40:23.106414 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 19 00:40:23.106414 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 19 00:40:23.106414 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 302, in validate Oct 19 00:40:23.106414 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) Oct 19 00:40:23.106414 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 303, in validate Oct 19 00:40:23.106414 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi raise error Oct 19 00:40:23.106414 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: '' does not match '^[0-9]+$' Oct 19 00:40:23.106414 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:40:23.106414 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Failed validating 'pattern' in schema['properties']['volume']['properties']['size']: Oct 19 00:40:23.106414 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi {'minimum': 1, 'pattern': '^[0-9]+$', 'type': ['integer', 'string']} Oct 19 00:40:23.106414 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:40:23.106414 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi On instance['volume']: Oct 19 00:40:23.106414 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi '' Oct 19 00:40:23.106414 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:40:23.106414 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 19 00:40:23.106414 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:40:23.106414 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 19 00:40:23.106414 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 19 00:40:23.106414 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 19 00:40:23.106414 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 19 00:40:23.106414 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 19 00:40:23.108034 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 19 00:40:23.108034 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 19 00:40:23.108034 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 19 00:40:23.108034 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 19 00:40:23.108034 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper Oct 19 00:40:23.108034 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], Oct 19 00:40:23.108034 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper Oct 19 00:40:23.108034 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) Oct 19 00:40:23.108034 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 331, in validate Oct 19 00:40:23.108034 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) Oct 19 00:40:23.108034 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute size. Value: . '' does not match '^[0-9]+$' Oct 19 00:40:23.108034 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:40:23.108034 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 19 00:40:23.108034 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:40:23.108034 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 19 00:40:23.108034 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 19 00:40:23.108034 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 19 00:40:23.108034 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ Oct 19 00:40:23.108034 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( Oct 19 00:40:23.109745 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute size. Value: . '' does not match '^[0-9]+$' Oct 19 00:40:23.109745 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:40:23.111277 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-95ffc4ba-76be-43c1-9bd2-48142d94f41c tempest-VolumesNegativeTest-1777702828 tempest-VolumesNegativeTest-1777702828-project] Returning 400 to user: Invalid input for field/attribute size. Value: . '' does not match '^[0-9]+$' {{(pid=125641) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 19 00:40:23.112334 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-95ffc4ba-76be-43c1-9bd2-48142d94f41c tempest-VolumesNegativeTest-1777702828 tempest-VolumesNegativeTest-1777702828-project] 10.222.0.26 "POST /compute/v2.1/os-volumes" status: 400 len: 121 microversion: 2.1 time: 0.020853 Oct 19 00:40:23.113688 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 2/3] 10.222.0.26 () {60 vars in 1156 bytes} [Wed Oct 19 00:40:23 2022] POST /compute/v2.1/os-volumes => generated 121 bytes in 23 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Oct 19 00:40:23.123198 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi [None req-dea2b7a1-bf85-4e35-9275-adada1e2e015 tempest-VolumesNegativeTest-1777702828 tempest-VolumesNegativeTest-1777702828-project] Calling method '.version_select at 0x7f71c8e87f70>' {{(pid=125640) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:40:23.153818 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG dbcounter [-] [125640] Writing DB stats nova_api:SELECT=3 {{(pid=125640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 00:40:23.153818 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG dbcounter [-] [125640] Writing DB stats nova_cell0:SELECT=3 {{(pid=125640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 00:40:23.153818 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG dbcounter [-] [125640] Writing DB stats nova_cell1:SELECT=3 {{(pid=125640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 00:40:23.154289 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG dbcounter [-] [125640] Writing DB stats nova_cell0:SELECT=3 {{(pid=125640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 00:40:24.267217 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: INFO nova.api.openstack.wsgi [None req-dea2b7a1-bf85-4e35-9275-adada1e2e015 tempest-VolumesNegativeTest-1777702828 tempest-VolumesNegativeTest-1777702828-project] HTTP exception thrown: Volume tempest-invalid-1555959773 could not be found. Oct 19 00:40:24.274108 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi [None req-dea2b7a1-bf85-4e35-9275-adada1e2e015 tempest-VolumesNegativeTest-1777702828 tempest-VolumesNegativeTest-1777702828-project] Request method failure captured: Oct 19 00:40:24.274108 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: request: DELETE /compute/v2.1/os-volumes/tempest-invalid-1555959773 HTTP/1.1 Oct 19 00:40:24.274108 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: Accept: application/json Oct 19 00:40:24.274108 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: Accept-Encoding: identity Oct 19 00:40:24.274108 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: Connection: close Oct 19 00:40:24.274108 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: Content-Length: 0 Oct 19 00:40:24.274108 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: Content-Type: application/json Oct 19 00:40:24.274108 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: Host: 10.222.0.26 Oct 19 00:40:24.274108 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: Openstack-System-Scope: None Oct 19 00:40:24.274108 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: User-Agent: python-urllib3/1.26.12 Oct 19 00:40:24.274108 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: X-Auth-Token: gAAAAABjT0d1daPsFvMJjSOTOV_G5OpgqDipoTmAzhu8K9YedqGdEfTFjeo5Tm_5A0vyaGOXa_JjRufNWIm5eeq3jVi34mVfYCxOeql1o4l3a11zoMxtDaoeiUXW9sPbOWgHA2J_SPeKzdHcB181eO-WC8M7UG0lX3mF62HWBLnwXvR9Npf1HTQ Oct 19 00:40:24.274108 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: X-Domain-Id: None Oct 19 00:40:24.274108 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: X-Domain-Name: None Oct 19 00:40:24.274108 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: X-Identity-Status: Confirmed Oct 19 00:40:24.274108 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: X-Is-Admin-Project: True Oct 19 00:40:24.274108 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: X-Project-Domain-Id: default Oct 19 00:40:24.274108 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: X-Project-Domain-Name: Default Oct 19 00:40:24.274108 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: X-Project-Id: 9e9333ca01214933af63450c15a2b51b Oct 19 00:40:24.274108 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: X-Project-Name: tempest-VolumesNegativeTest-1777702828 Oct 19 00:40:24.274108 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: X-Role: member,reader Oct 19 00:40:24.274108 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: X-Roles: member,reader Oct 19 00:40:24.275602 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/9e9333ca01214933af63450c15a2b51b"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/9e9333ca01214933af63450c15a2b51b"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/9e9333ca01214933af63450c15a2b51b"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_9e9333ca01214933af63450c15a2b51b"}]}] Oct 19 00:40:24.275602 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: X-Tenant: tempest-VolumesNegativeTest-1777702828 Oct 19 00:40:24.275602 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: X-Tenant-Id: 9e9333ca01214933af63450c15a2b51b Oct 19 00:40:24.275602 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: X-Tenant-Name: tempest-VolumesNegativeTest-1777702828 Oct 19 00:40:24.275602 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: X-User: tempest-VolumesNegativeTest-1777702828-project Oct 19 00:40:24.275602 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: X-User-Domain-Id: default Oct 19 00:40:24.275602 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: X-User-Domain-Name: Default Oct 19 00:40:24.275602 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: X-User-Id: e9740ef47b0145c18430a28c0e53a0d6 Oct 19 00:40:24.275602 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: X-User-Name: tempest-VolumesNegativeTest-1777702828-project Oct 19 00:40:24.275602 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: method: Oct 19 00:40:24.275602 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: exception: Volume tempest-invalid-1555959773 could not be found. Oct 19 00:40:24.275602 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: action_args: {'id': 'tempest-invalid-1555959773'} Oct 19 00:40:24.275602 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: {{(pid=125640) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 19 00:40:24.276475 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 19 00:40:24.276475 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Oct 19 00:40:24.276475 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Oct 19 00:40:24.276475 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete Oct 19 00:40:24.276475 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) Oct 19 00:40:24.276475 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete Oct 19 00:40:24.276475 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi return self._delete(loc) Oct 19 00:40:24.276475 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete Oct 19 00:40:24.276475 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) Oct 19 00:40:24.276475 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete Oct 19 00:40:24.276475 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) Oct 19 00:40:24.276475 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Oct 19 00:40:24.276475 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Oct 19 00:40:24.276475 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Oct 19 00:40:24.276475 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Oct 19 00:40:24.276475 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume tempest-invalid-1555959773 could not be found. (HTTP 404) (Request-ID: req-56f6e7ce-e19c-4239-bc58-3562eb6115a9) Oct 19 00:40:24.276475 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi Oct 19 00:40:24.276475 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 19 00:40:24.276475 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi Oct 19 00:40:24.278289 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 19 00:40:24.278289 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 136, in delete Oct 19 00:40:24.278289 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi self.volume_api.delete(context, id) Oct 19 00:40:24.278289 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Oct 19 00:40:24.278289 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Oct 19 00:40:24.278289 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Oct 19 00:40:24.278289 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Oct 19 00:40:24.278289 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Oct 19 00:40:24.278289 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Oct 19 00:40:24.278289 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Oct 19 00:40:24.278289 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Oct 19 00:40:24.278289 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete Oct 19 00:40:24.278289 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) Oct 19 00:40:24.278289 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete Oct 19 00:40:24.278289 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi return self._delete(loc) Oct 19 00:40:24.278289 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete Oct 19 00:40:24.278289 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) Oct 19 00:40:24.278289 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete Oct 19 00:40:24.278289 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) Oct 19 00:40:24.279927 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Oct 19 00:40:24.279927 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Oct 19 00:40:24.279927 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Oct 19 00:40:24.279927 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Oct 19 00:40:24.279927 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume tempest-invalid-1555959773 could not be found. Oct 19 00:40:24.279927 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi Oct 19 00:40:24.279927 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 19 00:40:24.279927 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi Oct 19 00:40:24.279927 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 19 00:40:24.279927 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 19 00:40:24.279927 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 19 00:40:24.279927 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 19 00:40:24.279927 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 19 00:40:24.279927 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 19 00:40:24.279927 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 19 00:40:24.279927 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 19 00:40:24.279927 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 19 00:40:24.279927 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 140, in delete Oct 19 00:40:24.279927 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 19 00:40:24.279927 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume tempest-invalid-1555959773 could not be found. Oct 19 00:40:24.283343 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi Oct 19 00:40:24.283343 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 19 00:40:24.283343 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi Oct 19 00:40:24.283343 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 19 00:40:24.283343 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 19 00:40:24.283343 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 19 00:40:24.283343 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 19 00:40:24.283343 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 19 00:40:24.283343 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume tempest-invalid-1555959773 could not be found. Oct 19 00:40:24.283343 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi Oct 19 00:40:24.283343 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi [None req-dea2b7a1-bf85-4e35-9275-adada1e2e015 tempest-VolumesNegativeTest-1777702828 tempest-VolumesNegativeTest-1777702828-project] Returning 404 to user: Volume tempest-invalid-1555959773 could not be found. {{(pid=125640) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 19 00:40:24.283343 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: INFO nova.api.openstack.requestlog [None req-dea2b7a1-bf85-4e35-9275-adada1e2e015 tempest-VolumesNegativeTest-1777702828 tempest-VolumesNegativeTest-1777702828-project] 10.222.0.26 "DELETE /compute/v2.1/os-volumes/tempest-invalid-1555959773" status: 404 len: 99 microversion: 2.1 time: 1.156421 Oct 19 00:40:24.285024 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: [pid: 125640|app: 0|req: 2/4] 10.222.0.26 () {58 vars in 1218 bytes} [Wed Oct 19 00:40:23 2022] DELETE /compute/v2.1/os-volumes/tempest-invalid-1555959773 => generated 99 bytes in 1165 msecs (HTTP/1.1 404) 9 headers in 378 bytes (1 switches on core 0) Oct 19 00:40:24.290633 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-8173c8df-60fa-4923-9ddd-17decb7e228e tempest-VolumesNegativeTest-1777702828 tempest-VolumesNegativeTest-1777702828-project] 10.222.0.26 "DELETE /compute/v2.1/os-volumes/" status: 404 len: 112 microversion: - time: 0.003746 Oct 19 00:40:24.291891 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 3/5] 10.222.0.26 () {58 vars in 1140 bytes} [Wed Oct 19 00:40:24 2022] DELETE /compute/v2.1/os-volumes/ => generated 112 bytes in 6 msecs (HTTP/1.1 404) 5 headers in 228 bytes (1 switches on core 0) Oct 19 00:40:24.312683 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: INFO nova.api.openstack.requestlog [None req-faf048dc-304a-45f0-8874-a94528a81f0e tempest-VolumesNegativeTest-1777702828 tempest-VolumesNegativeTest-1777702828-project] 10.222.0.26 "GET /compute/v2.1/os-volumes/" status: 404 len: 112 microversion: - time: 0.012994 Oct 19 00:40:24.328385 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-f490454f-6cfb-4ffb-9f7e-4d3e190709cb tempest-VolumesNegativeTest-1777702828 tempest-VolumesNegativeTest-1777702828-project] Calling method '.version_select at 0x7f71c8cffa60>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:40:24.334866 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: [pid: 125640|app: 0|req: 3/6] 10.222.0.26 () {58 vars in 1137 bytes} [Wed Oct 19 00:40:24 2022] GET /compute/v2.1/os-volumes/ => generated 112 bytes in 36 msecs (HTTP/1.1 404) 5 headers in 228 bytes (1 switches on core 0) Oct 19 00:40:24.802137 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.wsgi [None req-f490454f-6cfb-4ffb-9f7e-4d3e190709cb tempest-VolumesNegativeTest-1777702828 tempest-VolumesNegativeTest-1777702828-project] HTTP exception thrown: Volume 4c685e53-ed9d-4659-9cbe-5b2371d4fe6e could not be found. Oct 19 00:40:24.807565 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-f490454f-6cfb-4ffb-9f7e-4d3e190709cb tempest-VolumesNegativeTest-1777702828 tempest-VolumesNegativeTest-1777702828-project] Request method failure captured: Oct 19 00:40:24.807565 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: request: DELETE /compute/v2.1/os-volumes/4c685e53-ed9d-4659-9cbe-5b2371d4fe6e HTTP/1.1 Oct 19 00:40:24.807565 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Accept: application/json Oct 19 00:40:24.807565 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Accept-Encoding: identity Oct 19 00:40:24.807565 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Connection: close Oct 19 00:40:24.807565 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Content-Length: 0 Oct 19 00:40:24.807565 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Content-Type: application/json Oct 19 00:40:24.807565 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Host: 10.222.0.26 Oct 19 00:40:24.807565 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Openstack-System-Scope: None Oct 19 00:40:24.807565 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: User-Agent: python-urllib3/1.26.12 Oct 19 00:40:24.807565 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Auth-Token: gAAAAABjT0d1daPsFvMJjSOTOV_G5OpgqDipoTmAzhu8K9YedqGdEfTFjeo5Tm_5A0vyaGOXa_JjRufNWIm5eeq3jVi34mVfYCxOeql1o4l3a11zoMxtDaoeiUXW9sPbOWgHA2J_SPeKzdHcB181eO-WC8M7UG0lX3mF62HWBLnwXvR9Npf1HTQ Oct 19 00:40:24.807565 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Domain-Id: None Oct 19 00:40:24.807565 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Domain-Name: None Oct 19 00:40:24.807565 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Identity-Status: Confirmed Oct 19 00:40:24.807565 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Is-Admin-Project: True Oct 19 00:40:24.807565 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Project-Domain-Id: default Oct 19 00:40:24.807565 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Project-Domain-Name: Default Oct 19 00:40:24.807565 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Project-Id: 9e9333ca01214933af63450c15a2b51b Oct 19 00:40:24.807565 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Project-Name: tempest-VolumesNegativeTest-1777702828 Oct 19 00:40:24.807565 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Role: member,reader Oct 19 00:40:24.807565 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Roles: member,reader Oct 19 00:40:24.809224 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/9e9333ca01214933af63450c15a2b51b"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/9e9333ca01214933af63450c15a2b51b"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/9e9333ca01214933af63450c15a2b51b"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_9e9333ca01214933af63450c15a2b51b"}]}] Oct 19 00:40:24.809224 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Tenant: tempest-VolumesNegativeTest-1777702828 Oct 19 00:40:24.809224 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Tenant-Id: 9e9333ca01214933af63450c15a2b51b Oct 19 00:40:24.809224 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Tenant-Name: tempest-VolumesNegativeTest-1777702828 Oct 19 00:40:24.809224 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User: tempest-VolumesNegativeTest-1777702828-project Oct 19 00:40:24.809224 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User-Domain-Id: default Oct 19 00:40:24.809224 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User-Domain-Name: Default Oct 19 00:40:24.809224 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User-Id: e9740ef47b0145c18430a28c0e53a0d6 Oct 19 00:40:24.809224 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User-Name: tempest-VolumesNegativeTest-1777702828-project Oct 19 00:40:24.809224 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: method: Oct 19 00:40:24.809224 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: exception: Volume 4c685e53-ed9d-4659-9cbe-5b2371d4fe6e could not be found. Oct 19 00:40:24.809224 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: action_args: {'id': '4c685e53-ed9d-4659-9cbe-5b2371d4fe6e'} Oct 19 00:40:24.809224 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 19 00:40:24.810207 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 19 00:40:24.810207 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Oct 19 00:40:24.810207 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Oct 19 00:40:24.810207 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete Oct 19 00:40:24.810207 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) Oct 19 00:40:24.810207 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete Oct 19 00:40:24.810207 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi return self._delete(loc) Oct 19 00:40:24.810207 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete Oct 19 00:40:24.810207 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) Oct 19 00:40:24.810207 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete Oct 19 00:40:24.810207 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) Oct 19 00:40:24.810207 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Oct 19 00:40:24.810207 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Oct 19 00:40:24.810207 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Oct 19 00:40:24.810207 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Oct 19 00:40:24.810207 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume 4c685e53-ed9d-4659-9cbe-5b2371d4fe6e could not be found. (HTTP 404) (Request-ID: req-658b1b83-94e1-4585-95de-bdbdb970124e) Oct 19 00:40:24.810207 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:40:24.810207 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 19 00:40:24.810207 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:40:24.811566 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 19 00:40:24.811566 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 136, in delete Oct 19 00:40:24.811566 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi self.volume_api.delete(context, id) Oct 19 00:40:24.811566 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Oct 19 00:40:24.811566 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Oct 19 00:40:24.811566 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Oct 19 00:40:24.811566 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Oct 19 00:40:24.811566 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Oct 19 00:40:24.811566 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Oct 19 00:40:24.811566 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Oct 19 00:40:24.811566 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Oct 19 00:40:24.811566 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete Oct 19 00:40:24.811566 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) Oct 19 00:40:24.811566 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete Oct 19 00:40:24.811566 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi return self._delete(loc) Oct 19 00:40:24.811566 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete Oct 19 00:40:24.811566 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) Oct 19 00:40:24.811566 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete Oct 19 00:40:24.811566 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) Oct 19 00:40:24.812925 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Oct 19 00:40:24.812925 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Oct 19 00:40:24.812925 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Oct 19 00:40:24.812925 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Oct 19 00:40:24.812925 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume 4c685e53-ed9d-4659-9cbe-5b2371d4fe6e could not be found. Oct 19 00:40:24.812925 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:40:24.812925 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 19 00:40:24.812925 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:40:24.812925 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 19 00:40:24.812925 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 19 00:40:24.812925 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 19 00:40:24.812925 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 19 00:40:24.812925 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 19 00:40:24.812925 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 19 00:40:24.812925 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 19 00:40:24.812925 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 19 00:40:24.812925 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 19 00:40:24.812925 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 140, in delete Oct 19 00:40:24.812925 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 19 00:40:24.814262 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume 4c685e53-ed9d-4659-9cbe-5b2371d4fe6e could not be found. Oct 19 00:40:24.814262 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:40:24.814262 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 19 00:40:24.814262 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:40:24.814262 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 19 00:40:24.814262 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 19 00:40:24.814262 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 19 00:40:24.814262 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 19 00:40:24.814262 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 19 00:40:24.814262 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume 4c685e53-ed9d-4659-9cbe-5b2371d4fe6e could not be found. Oct 19 00:40:24.814262 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:40:24.816987 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-f490454f-6cfb-4ffb-9f7e-4d3e190709cb tempest-VolumesNegativeTest-1777702828 tempest-VolumesNegativeTest-1777702828-project] Returning 404 to user: Volume 4c685e53-ed9d-4659-9cbe-5b2371d4fe6e could not be found. {{(pid=125641) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 19 00:40:24.819071 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-f490454f-6cfb-4ffb-9f7e-4d3e190709cb tempest-VolumesNegativeTest-1777702828 tempest-VolumesNegativeTest-1777702828-project] 10.222.0.26 "DELETE /compute/v2.1/os-volumes/4c685e53-ed9d-4659-9cbe-5b2371d4fe6e" status: 404 len: 109 microversion: 2.1 time: 0.494306 Oct 19 00:40:24.834425 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi [None req-c3e5a4d1-a208-49e5-aa3f-1dffd5bbef65 tempest-VolumesNegativeTest-1777702828 tempest-VolumesNegativeTest-1777702828-project] Calling method '.version_select at 0x7f71c8c2e700>' {{(pid=125640) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:40:24.840499 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 4/7] 10.222.0.26 () {58 vars in 1248 bytes} [Wed Oct 19 00:40:24 2022] DELETE /compute/v2.1/os-volumes/4c685e53-ed9d-4659-9cbe-5b2371d4fe6e => generated 109 bytes in 516 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 19 00:40:24.979853 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: INFO nova.api.openstack.wsgi [None req-c3e5a4d1-a208-49e5-aa3f-1dffd5bbef65 tempest-VolumesNegativeTest-1777702828 tempest-VolumesNegativeTest-1777702828-project] HTTP exception thrown: Volume f631f316-3d3a-497c-a66f-9b8d2b63f1ae could not be found. Oct 19 00:40:24.982944 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi [None req-c3e5a4d1-a208-49e5-aa3f-1dffd5bbef65 tempest-VolumesNegativeTest-1777702828 tempest-VolumesNegativeTest-1777702828-project] Request method failure captured: Oct 19 00:40:24.982944 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: request: GET /compute/v2.1/os-volumes/f631f316-3d3a-497c-a66f-9b8d2b63f1ae HTTP/1.1 Oct 19 00:40:24.982944 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: Accept: application/json Oct 19 00:40:24.982944 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: Accept-Encoding: identity Oct 19 00:40:24.982944 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: Connection: close Oct 19 00:40:24.982944 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: Content-Length: 0 Oct 19 00:40:24.982944 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: Content-Type: application/json Oct 19 00:40:24.982944 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: Host: 10.222.0.26 Oct 19 00:40:24.982944 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: Openstack-System-Scope: None Oct 19 00:40:24.982944 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: User-Agent: python-urllib3/1.26.12 Oct 19 00:40:24.982944 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: X-Auth-Token: gAAAAABjT0d1daPsFvMJjSOTOV_G5OpgqDipoTmAzhu8K9YedqGdEfTFjeo5Tm_5A0vyaGOXa_JjRufNWIm5eeq3jVi34mVfYCxOeql1o4l3a11zoMxtDaoeiUXW9sPbOWgHA2J_SPeKzdHcB181eO-WC8M7UG0lX3mF62HWBLnwXvR9Npf1HTQ Oct 19 00:40:24.982944 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: X-Domain-Id: None Oct 19 00:40:24.982944 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: X-Domain-Name: None Oct 19 00:40:24.982944 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: X-Identity-Status: Confirmed Oct 19 00:40:24.982944 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: X-Is-Admin-Project: True Oct 19 00:40:24.982944 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: X-Project-Domain-Id: default Oct 19 00:40:24.982944 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: X-Project-Domain-Name: Default Oct 19 00:40:24.982944 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: X-Project-Id: 9e9333ca01214933af63450c15a2b51b Oct 19 00:40:24.982944 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: X-Project-Name: tempest-VolumesNegativeTest-1777702828 Oct 19 00:40:24.982944 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: X-Role: member,reader Oct 19 00:40:24.982944 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: X-Roles: member,reader Oct 19 00:40:24.984570 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/9e9333ca01214933af63450c15a2b51b"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/9e9333ca01214933af63450c15a2b51b"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/9e9333ca01214933af63450c15a2b51b"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_9e9333ca01214933af63450c15a2b51b"}]}] Oct 19 00:40:24.984570 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: X-Tenant: tempest-VolumesNegativeTest-1777702828 Oct 19 00:40:24.984570 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: X-Tenant-Id: 9e9333ca01214933af63450c15a2b51b Oct 19 00:40:24.984570 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: X-Tenant-Name: tempest-VolumesNegativeTest-1777702828 Oct 19 00:40:24.984570 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: X-User: tempest-VolumesNegativeTest-1777702828-project Oct 19 00:40:24.984570 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: X-User-Domain-Id: default Oct 19 00:40:24.984570 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: X-User-Domain-Name: Default Oct 19 00:40:24.984570 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: X-User-Id: e9740ef47b0145c18430a28c0e53a0d6 Oct 19 00:40:24.984570 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: X-User-Name: tempest-VolumesNegativeTest-1777702828-project Oct 19 00:40:24.984570 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: method: Oct 19 00:40:24.984570 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: exception: Volume f631f316-3d3a-497c-a66f-9b8d2b63f1ae could not be found. Oct 19 00:40:24.984570 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: action_args: {'id': 'f631f316-3d3a-497c-a66f-9b8d2b63f1ae'} Oct 19 00:40:24.984570 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: {{(pid=125640) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 19 00:40:24.985527 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 19 00:40:24.985527 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Oct 19 00:40:24.985527 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Oct 19 00:40:24.985527 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Oct 19 00:40:24.985527 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi item = cinderclient( Oct 19 00:40:24.985527 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Oct 19 00:40:24.985527 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Oct 19 00:40:24.985527 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Oct 19 00:40:24.985527 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Oct 19 00:40:24.985527 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Oct 19 00:40:24.985527 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Oct 19 00:40:24.985527 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Oct 19 00:40:24.985527 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Oct 19 00:40:24.985527 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Oct 19 00:40:24.985527 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Oct 19 00:40:24.985527 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume f631f316-3d3a-497c-a66f-9b8d2b63f1ae could not be found. (HTTP 404) (Request-ID: req-22f62c1a-a580-4c98-82ac-eb64e8f2933a) Oct 19 00:40:24.985527 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi Oct 19 00:40:24.985527 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 19 00:40:24.985527 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi Oct 19 00:40:24.990244 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 19 00:40:24.990244 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 120, in show Oct 19 00:40:24.990244 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi vol = self.volume_api.get(context, id) Oct 19 00:40:24.990244 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Oct 19 00:40:24.990244 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Oct 19 00:40:24.990244 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Oct 19 00:40:24.990244 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Oct 19 00:40:24.990244 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Oct 19 00:40:24.990244 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Oct 19 00:40:24.990244 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Oct 19 00:40:24.990244 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Oct 19 00:40:24.990244 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Oct 19 00:40:24.990244 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi item = cinderclient( Oct 19 00:40:24.990244 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Oct 19 00:40:24.990244 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Oct 19 00:40:24.990244 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Oct 19 00:40:24.990244 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Oct 19 00:40:24.990244 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Oct 19 00:40:24.990244 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Oct 19 00:40:24.999613 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Oct 19 00:40:24.999613 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Oct 19 00:40:24.999613 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Oct 19 00:40:24.999613 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Oct 19 00:40:24.999613 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume f631f316-3d3a-497c-a66f-9b8d2b63f1ae could not be found. Oct 19 00:40:24.999613 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi Oct 19 00:40:24.999613 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 19 00:40:24.999613 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi Oct 19 00:40:24.999613 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 19 00:40:24.999613 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 19 00:40:24.999613 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 19 00:40:24.999613 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 19 00:40:24.999613 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 19 00:40:24.999613 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 19 00:40:24.999613 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 19 00:40:24.999613 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 19 00:40:24.999613 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 19 00:40:24.999613 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 122, in show Oct 19 00:40:24.999613 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 19 00:40:25.001950 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume f631f316-3d3a-497c-a66f-9b8d2b63f1ae could not be found. Oct 19 00:40:25.001950 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi Oct 19 00:40:25.001950 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 19 00:40:25.001950 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi Oct 19 00:40:25.001950 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 19 00:40:25.001950 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 19 00:40:25.001950 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 19 00:40:25.001950 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 19 00:40:25.001950 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 19 00:40:25.001950 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume f631f316-3d3a-497c-a66f-9b8d2b63f1ae could not be found. Oct 19 00:40:25.001950 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi Oct 19 00:40:25.022613 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi [None req-c3e5a4d1-a208-49e5-aa3f-1dffd5bbef65 tempest-VolumesNegativeTest-1777702828 tempest-VolumesNegativeTest-1777702828-project] Returning 404 to user: Volume f631f316-3d3a-497c-a66f-9b8d2b63f1ae could not be found. {{(pid=125640) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 19 00:40:25.024307 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: INFO nova.api.openstack.requestlog [None req-c3e5a4d1-a208-49e5-aa3f-1dffd5bbef65 tempest-VolumesNegativeTest-1777702828 tempest-VolumesNegativeTest-1777702828-project] 10.222.0.26 "GET /compute/v2.1/os-volumes/f631f316-3d3a-497c-a66f-9b8d2b63f1ae" status: 404 len: 109 microversion: 2.1 time: 0.192856 Oct 19 00:40:25.026716 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: [pid: 125640|app: 0|req: 4/8] 10.222.0.26 () {58 vars in 1245 bytes} [Wed Oct 19 00:40:24 2022] GET /compute/v2.1/os-volumes/f631f316-3d3a-497c-a66f-9b8d2b63f1ae => generated 109 bytes in 196 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 19 00:40:39.277814 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-bc84d98e-8d85-4a34-947f-a6f61e216954 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerRescueNegativeTestJSON-server-92757836", "imageRef": "b387b4f8-f75e-4a68-b49c-a015bf61c46b", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "}]}} {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 19 00:40:39.522376 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.network.neutron [None req-bc84d98e-8d85-4a34-947f-a6f61e216954 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] validate_networks() for [('75c84943-00ac-41e3-a6c3-2ea734b46148', None, None, None, None, None)] {{(pid=125641) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 19 00:40:40.159733 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi [None req-3f8f4007-1dd0-425c-b725-53f09d98b065 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Action: 'create', calling method: .version_select at 0x7f71c8c2e700>, body: {"keypair": {"name": "tempest-keypair-2045940183", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBE1BZfVtDg4vf2oQXpiderZ9D7cNY6hGjj325Rsxf9rJriGPFkb47PSRfaNGRBocdAjp0rjPZtGYrwCSaZEiVoFUqhbi1zFr8pd6AjhQIf6n1md2my2SlcabfPQ7XwaDvw=="}} {{(pid=125640) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 19 00:40:40.304629 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.quota [None req-3f8f4007-1dd0-425c-b725-53f09d98b065 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Getting quotas for project 37ffcffcae9b41bbb2d13adc49cb99df. Resources: {'key_pairs'} {{(pid=125640) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 19 00:40:40.329129 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.quota [None req-3f8f4007-1dd0-425c-b725-53f09d98b065 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Getting quotas for user 69fef5650fff42e4ac4ed3ca74d4b123 and project 37ffcffcae9b41bbb2d13adc49cb99df. Resources: {'key_pairs'} {{(pid=125640) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 19 00:40:40.415987 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: INFO nova.api.openstack.requestlog [None req-3f8f4007-1dd0-425c-b725-53f09d98b065 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] 10.222.0.26 "POST /compute/v2.1/os-keypairs" status: 201 len: 401 microversion: 2.20 time: 0.648711 Oct 19 00:40:40.416788 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: [pid: 125640|app: 0|req: 5/9] 10.222.0.26 () {62 vars in 1200 bytes} [Wed Oct 19 00:40:39 2022] POST /compute/v2.1/os-keypairs => generated 401 bytes in 652 msecs (HTTP/1.1 201) 9 headers in 364 bytes (2 switches on core 0) Oct 19 00:40:42.355236 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi [None req-aae593cc-2394-43ed-b81c-6f3e6ff97eb3 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Action: 'create', calling method: .version_select at 0x7f71c8c151f0>, body: {"keypair": {"name": "tempest-keypair-80676479", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBFUgovPOkxfhK+qmwpKNU5CZWQaeT7pa13m4Z8NXGj0hEGeJINwK33wzwFnPfpHlBDtEgzDFxPM7r7W6C8EE1N3osoSr+CgQIDs2+U+rvh1CmtT+9ojigvU6YTZohdk/tg=="}} {{(pid=125640) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 19 00:40:42.373311 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.quota [None req-aae593cc-2394-43ed-b81c-6f3e6ff97eb3 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Getting quotas for project 0b35c2038f6b4d0a9012b004f1416c00. Resources: {'key_pairs'} {{(pid=125640) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 19 00:40:42.397184 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.quota [None req-aae593cc-2394-43ed-b81c-6f3e6ff97eb3 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Getting quotas for user 6f8725c4d05b4dd79b05b43fba15980b and project 0b35c2038f6b4d0a9012b004f1416c00. Resources: {'key_pairs'} {{(pid=125640) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 19 00:40:42.472802 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: INFO nova.api.openstack.requestlog [None req-aae593cc-2394-43ed-b81c-6f3e6ff97eb3 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] 10.222.0.26 "POST /compute/v2.1/os-keypairs" status: 200 len: 384 microversion: 2.1 time: 0.771276 Oct 19 00:40:42.473753 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: [pid: 125640|app: 0|req: 6/10] 10.222.0.26 () {60 vars in 1159 bytes} [Wed Oct 19 00:40:41 2022] POST /compute/v2.1/os-keypairs => generated 384 bytes in 773 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 19 00:40:43.055230 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi [None req-1d523caf-a828-4870-acab-932bda5e6fb2 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-DeleteServersTestJSON-server-676135712", "imageRef": "b387b4f8-f75e-4a68-b49c-a015bf61c46b", "flavorRef": "452", "networks": [{"uuid": "db3956ab-e454-4c4b-8ece-d9089b41eb0d"}]}} {{(pid=125640) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 19 00:40:43.220130 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.network.neutron [None req-1d523caf-a828-4870-acab-932bda5e6fb2 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] validate_networks() for [('db3956ab-e454-4c4b-8ece-d9089b41eb0d', None, None, None, None, None)] {{(pid=125640) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 19 00:40:43.672237 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.virt.hardware [None req-bc84d98e-8d85-4a34-947f-a6f61e216954 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Flavor limits 0:0:0 {{(pid=125641) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 19 00:40:43.672834 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.virt.hardware [None req-bc84d98e-8d85-4a34-947f-a6f61e216954 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Image limits 0:0:0 {{(pid=125641) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 19 00:40:43.674073 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.virt.hardware [None req-bc84d98e-8d85-4a34-947f-a6f61e216954 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Flavor pref 0:0:0 {{(pid=125641) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 19 00:40:43.674073 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.virt.hardware [None req-bc84d98e-8d85-4a34-947f-a6f61e216954 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Image pref 0:0:0 {{(pid=125641) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 19 00:40:43.674073 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.virt.hardware [None req-bc84d98e-8d85-4a34-947f-a6f61e216954 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=125641) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 19 00:40:43.707317 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-bc84d98e-8d85-4a34-947f-a6f61e216954 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:40:43.709131 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-bc84d98e-8d85-4a34-947f-a6f61e216954 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:40:43.710708 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-bc84d98e-8d85-4a34-947f-a6f61e216954 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:40:43.711615 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-bc84d98e-8d85-4a34-947f-a6f61e216954 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:40:43.713118 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-bc84d98e-8d85-4a34-947f-a6f61e216954 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:40:43.715226 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-bc84d98e-8d85-4a34-947f-a6f61e216954 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:40:43.796744 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.quota [None req-bc84d98e-8d85-4a34-947f-a6f61e216954 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Getting quotas for project c0151d58abd047a692f5cc4e7d05bea5. Resources: {'cores', 'ram', 'instances'} {{(pid=125641) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 19 00:40:43.810965 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.quota [None req-bc84d98e-8d85-4a34-947f-a6f61e216954 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Getting quotas for user d3f0d615187d4779bdd7da8c9f24c796 and project c0151d58abd047a692f5cc4e7d05bea5. Resources: {'cores', 'ram', 'instances'} {{(pid=125641) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 19 00:40:43.839640 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-bc84d98e-8d85-4a34-947f-a6f61e216954 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Going to run 1 instances... {{(pid=125641) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 19 00:40:43.842942 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-bc84d98e-8d85-4a34-947f-a6f61e216954 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] [instance: 3da61834-fd67-49d2-925c-c18fc264042c] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='b387b4f8-f75e-4a68-b49c-a015bf61c46b',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=125641) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 19 00:40:43.945262 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-bc84d98e-8d85-4a34-947f-a6f61e216954 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 202 len: 383 microversion: 2.1 time: 5.221844 Oct 19 00:40:43.957885 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 5/11] 10.222.0.26 () {60 vars in 1147 bytes} [Wed Oct 19 00:40:38 2022] POST /compute/v2.1/servers => generated 383 bytes in 5233 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 19 00:40:43.966570 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-46f89080-1c74-4b5f-8cc3-0db3fcbe636d tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:40:43.967192 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-46f89080-1c74-4b5f-8cc3-0db3fcbe636d tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] [instance: 3da61834-fd67-49d2-925c-c18fc264042c] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:40:44.275038 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-46f89080-1c74-4b5f-8cc3-0db3fcbe636d tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd3f0d615187d4779bdd7da8c9f24c796', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0151d58abd047a692f5cc4e7d05bea5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:40:44.297308 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-46f89080-1c74-4b5f-8cc3-0db3fcbe636d tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] 10.222.0.26 "GET /compute/v2.1/servers/3da61834-fd67-49d2-925c-c18fc264042c" status: 200 len: 1220 microversion: 2.1 time: 0.337471 Oct 19 00:40:44.303580 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 6/12] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:40:43 2022] GET /compute/v2.1/servers/3da61834-fd67-49d2-925c-c18fc264042c => generated 1220 bytes in 345 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:40:44.646213 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-8443d0ac-f4c3-46c1-a58e-7bf310aa7a0c tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerRescueNegativeTestJSON-server-1514975080", "imageRef": "b387b4f8-f75e-4a68-b49c-a015bf61c46b", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "}]}} {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 19 00:40:44.807864 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.network.neutron [None req-8443d0ac-f4c3-46c1-a58e-7bf310aa7a0c tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] validate_networks() for [('75c84943-00ac-41e3-a6c3-2ea734b46148', None, None, None, None, None)] {{(pid=125641) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 19 00:40:45.802147 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.virt.hardware [None req-8443d0ac-f4c3-46c1-a58e-7bf310aa7a0c tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Flavor limits 0:0:0 {{(pid=125641) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 19 00:40:45.803661 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.virt.hardware [None req-8443d0ac-f4c3-46c1-a58e-7bf310aa7a0c tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Image limits 0:0:0 {{(pid=125641) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 19 00:40:45.804424 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.virt.hardware [None req-8443d0ac-f4c3-46c1-a58e-7bf310aa7a0c tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Flavor pref 0:0:0 {{(pid=125641) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 19 00:40:45.805611 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.virt.hardware [None req-8443d0ac-f4c3-46c1-a58e-7bf310aa7a0c tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Image pref 0:0:0 {{(pid=125641) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 19 00:40:45.806316 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.virt.hardware [None req-8443d0ac-f4c3-46c1-a58e-7bf310aa7a0c tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=125641) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 19 00:40:45.820283 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-8443d0ac-f4c3-46c1-a58e-7bf310aa7a0c tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:40:45.824892 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-8443d0ac-f4c3-46c1-a58e-7bf310aa7a0c tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:40:45.825622 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-8443d0ac-f4c3-46c1-a58e-7bf310aa7a0c tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:40:45.826554 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-8443d0ac-f4c3-46c1-a58e-7bf310aa7a0c tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:40:45.827252 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-8443d0ac-f4c3-46c1-a58e-7bf310aa7a0c tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:40:45.828849 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-8443d0ac-f4c3-46c1-a58e-7bf310aa7a0c tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:40:45.892209 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.quota [None req-8443d0ac-f4c3-46c1-a58e-7bf310aa7a0c tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Getting quotas for project c0151d58abd047a692f5cc4e7d05bea5. Resources: {'cores', 'ram', 'instances'} {{(pid=125641) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 19 00:40:45.898012 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.quota [None req-8443d0ac-f4c3-46c1-a58e-7bf310aa7a0c tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Getting quotas for user d3f0d615187d4779bdd7da8c9f24c796 and project c0151d58abd047a692f5cc4e7d05bea5. Resources: {'cores', 'ram', 'instances'} {{(pid=125641) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 19 00:40:45.930418 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-8443d0ac-f4c3-46c1-a58e-7bf310aa7a0c tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Going to run 1 instances... {{(pid=125641) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 19 00:40:45.933567 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-8443d0ac-f4c3-46c1-a58e-7bf310aa7a0c tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] [instance: ab4c73d6-9203-4110-93af-4a78428c672b] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='b387b4f8-f75e-4a68-b49c-a015bf61c46b',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=125641) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 19 00:40:46.057879 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-8443d0ac-f4c3-46c1-a58e-7bf310aa7a0c tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 202 len: 383 microversion: 2.1 time: 1.417753 Oct 19 00:40:46.062936 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 7/13] 10.222.0.26 () {60 vars in 1147 bytes} [Wed Oct 19 00:40:44 2022] POST /compute/v2.1/servers => generated 383 bytes in 1423 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 19 00:40:46.073873 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-d177f1c8-52f3-49b8-8e67-8fb866b28204 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeShelveTestJSON-server-481322472", "imageRef": "b387b4f8-f75e-4a68-b49c-a015bf61c46b", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "173cfaec-0c5f-4ee8-a8f2-5a3596dcd7bc"}], "security_groups": [{"name": "tempest-securitygroup--1017006986"}], "key_name": "tempest-keypair-2045940183", "user_data": "}} {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 19 00:40:46.406806 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.virt.hardware [None req-1d523caf-a828-4870-acab-932bda5e6fb2 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Flavor limits 0:0:0 {{(pid=125640) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 19 00:40:46.407209 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.virt.hardware [None req-1d523caf-a828-4870-acab-932bda5e6fb2 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Image limits 0:0:0 {{(pid=125640) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 19 00:40:46.407629 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.virt.hardware [None req-1d523caf-a828-4870-acab-932bda5e6fb2 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Flavor pref 0:0:0 {{(pid=125640) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 19 00:40:46.408144 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.virt.hardware [None req-1d523caf-a828-4870-acab-932bda5e6fb2 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Image pref 0:0:0 {{(pid=125640) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 19 00:40:46.409193 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.virt.hardware [None req-1d523caf-a828-4870-acab-932bda5e6fb2 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=125640) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 19 00:40:46.454337 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-1d523caf-a828-4870-acab-932bda5e6fb2 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:40:46.460384 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-1d523caf-a828-4870-acab-932bda5e6fb2 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:40:46.460855 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-1d523caf-a828-4870-acab-932bda5e6fb2 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:40:46.463869 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-1d523caf-a828-4870-acab-932bda5e6fb2 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:40:46.464349 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-1d523caf-a828-4870-acab-932bda5e6fb2 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:40:46.464784 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-1d523caf-a828-4870-acab-932bda5e6fb2 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:40:46.607737 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.quota [None req-1d523caf-a828-4870-acab-932bda5e6fb2 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Getting quotas for project f249d9041c6749f58973a5b0867b07ce. Resources: {'cores', 'ram', 'instances'} {{(pid=125640) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 19 00:40:46.631038 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.quota [None req-1d523caf-a828-4870-acab-932bda5e6fb2 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Getting quotas for user 8118f0c698aa428a93cb054abac30c2e and project f249d9041c6749f58973a5b0867b07ce. Resources: {'cores', 'ram', 'instances'} {{(pid=125640) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 19 00:40:46.710761 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.compute.api [None req-1d523caf-a828-4870-acab-932bda5e6fb2 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Going to run 1 instances... {{(pid=125640) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 19 00:40:46.713865 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.compute.api [None req-1d523caf-a828-4870-acab-932bda5e6fb2 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] [instance: e8a28aad-697f-4dee-8254-6f08e9d76168] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='b387b4f8-f75e-4a68-b49c-a015bf61c46b',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=125640) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 19 00:40:46.964673 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: INFO nova.api.openstack.requestlog [None req-1d523caf-a828-4870-acab-932bda5e6fb2 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 4.378862 Oct 19 00:40:46.965356 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: [pid: 125640|app: 0|req: 7/14] 10.222.0.26 () {60 vars in 1147 bytes} [Wed Oct 19 00:40:42 2022] POST /compute/v2.1/servers => generated 380 bytes in 4382 msecs (HTTP/1.1 202) 10 headers in 451 bytes (2 switches on core 0) Oct 19 00:40:47.003312 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi [None req-14ae08e9-ff14-4190-845f-9d478ca171fd tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Calling method '>' {{(pid=125640) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:40:47.004668 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.compute.api [None req-14ae08e9-ff14-4190-845f-9d478ca171fd tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] [instance: ab4c73d6-9203-4110-93af-4a78428c672b] Fetching instance by UUID {{(pid=125640) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:40:47.058979 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.network.neutron [None req-d177f1c8-52f3-49b8-8e67-8fb866b28204 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] validate_networks() for [('173cfaec-0c5f-4ee8-a8f2-5a3596dcd7bc', None, None, None, None, None)] {{(pid=125641) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 19 00:40:47.314258 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.policy [None req-14ae08e9-ff14-4190-845f-9d478ca171fd tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd3f0d615187d4779bdd7da8c9f24c796', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0151d58abd047a692f5cc4e7d05bea5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125640) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:40:47.351953 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: INFO nova.api.openstack.requestlog [None req-14ae08e9-ff14-4190-845f-9d478ca171fd tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] 10.222.0.26 "GET /compute/v2.1/servers/ab4c73d6-9203-4110-93af-4a78428c672b" status: 200 len: 1222 microversion: 2.1 time: 0.374205 Oct 19 00:40:47.355042 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: [pid: 125640|app: 0|req: 8/15] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:40:46 2022] GET /compute/v2.1/servers/ab4c73d6-9203-4110-93af-4a78428c672b => generated 1222 bytes in 389 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:40:47.384513 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi [None req-cab88ec1-3a7a-4264-ac3b-4baa114e8b0e tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Calling method '>' {{(pid=125640) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:40:47.386476 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.compute.api [None req-cab88ec1-3a7a-4264-ac3b-4baa114e8b0e tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] [instance: e8a28aad-697f-4dee-8254-6f08e9d76168] Fetching instance by UUID {{(pid=125640) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:40:47.672972 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.policy [None req-cab88ec1-3a7a-4264-ac3b-4baa114e8b0e tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8118f0c698aa428a93cb054abac30c2e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f249d9041c6749f58973a5b0867b07ce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125640) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:40:47.692748 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: INFO nova.api.openstack.requestlog [None req-cab88ec1-3a7a-4264-ac3b-4baa114e8b0e tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] 10.222.0.26 "GET /compute/v2.1/servers/e8a28aad-697f-4dee-8254-6f08e9d76168" status: 200 len: 1214 microversion: 2.1 time: 0.333101 Oct 19 00:40:47.695388 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: [pid: 125640|app: 0|req: 9/16] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:40:47 2022] GET /compute/v2.1/servers/e8a28aad-697f-4dee-8254-6f08e9d76168 => generated 1214 bytes in 339 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:40:48.378117 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi [None req-312341bc-f44f-453e-b241-1111b7f42f68 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Calling method '>' {{(pid=125640) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:40:48.378858 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.compute.api [None req-312341bc-f44f-453e-b241-1111b7f42f68 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] [instance: ab4c73d6-9203-4110-93af-4a78428c672b] Fetching instance by UUID {{(pid=125640) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:40:48.388268 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-312341bc-f44f-453e-b241-1111b7f42f68 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:40:48.391191 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-312341bc-f44f-453e-b241-1111b7f42f68 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:40:48.394947 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-312341bc-f44f-453e-b241-1111b7f42f68 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:40:48.395995 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-312341bc-f44f-453e-b241-1111b7f42f68 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:40:48.396651 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-312341bc-f44f-453e-b241-1111b7f42f68 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:40:48.398875 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-312341bc-f44f-453e-b241-1111b7f42f68 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:40:48.500877 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.virt.hardware [None req-d177f1c8-52f3-49b8-8e67-8fb866b28204 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Flavor limits 0:0:0 {{(pid=125641) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 19 00:40:48.501461 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.virt.hardware [None req-d177f1c8-52f3-49b8-8e67-8fb866b28204 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Image limits 0:0:0 {{(pid=125641) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 19 00:40:48.502236 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.virt.hardware [None req-d177f1c8-52f3-49b8-8e67-8fb866b28204 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Flavor pref 0:0:0 {{(pid=125641) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 19 00:40:48.502890 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.virt.hardware [None req-d177f1c8-52f3-49b8-8e67-8fb866b28204 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Image pref 0:0:0 {{(pid=125641) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 19 00:40:48.503422 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.virt.hardware [None req-d177f1c8-52f3-49b8-8e67-8fb866b28204 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=125641) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 19 00:40:48.524517 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d177f1c8-52f3-49b8-8e67-8fb866b28204 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:40:48.525186 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d177f1c8-52f3-49b8-8e67-8fb866b28204 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:40:48.525738 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d177f1c8-52f3-49b8-8e67-8fb866b28204 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:40:48.526738 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d177f1c8-52f3-49b8-8e67-8fb866b28204 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:40:48.527366 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d177f1c8-52f3-49b8-8e67-8fb866b28204 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:40:48.531201 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d177f1c8-52f3-49b8-8e67-8fb866b28204 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:40:48.626167 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.quota [None req-d177f1c8-52f3-49b8-8e67-8fb866b28204 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Getting quotas for project 37ffcffcae9b41bbb2d13adc49cb99df. Resources: {'cores', 'ram', 'instances'} {{(pid=125641) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 19 00:40:48.636208 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.quota [None req-d177f1c8-52f3-49b8-8e67-8fb866b28204 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Getting quotas for user 69fef5650fff42e4ac4ed3ca74d4b123 and project 37ffcffcae9b41bbb2d13adc49cb99df. Resources: {'cores', 'ram', 'instances'} {{(pid=125641) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 19 00:40:48.671370 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-d177f1c8-52f3-49b8-8e67-8fb866b28204 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Going to run 1 instances... {{(pid=125641) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 19 00:40:48.675051 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-d177f1c8-52f3-49b8-8e67-8fb866b28204 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] [instance: b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='b387b4f8-f75e-4a68-b49c-a015bf61c46b',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=125641) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 19 00:40:48.713882 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.policy [None req-312341bc-f44f-453e-b241-1111b7f42f68 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd3f0d615187d4779bdd7da8c9f24c796', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0151d58abd047a692f5cc4e7d05bea5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125640) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:40:48.749426 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: INFO nova.api.openstack.requestlog [None req-312341bc-f44f-453e-b241-1111b7f42f68 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] 10.222.0.26 "GET /compute/v2.1/servers/ab4c73d6-9203-4110-93af-4a78428c672b" status: 200 len: 1218 microversion: 2.1 time: 0.376024 Oct 19 00:40:48.750019 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: [pid: 125640|app: 0|req: 10/17] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:40:48 2022] GET /compute/v2.1/servers/ab4c73d6-9203-4110-93af-4a78428c672b => generated 1218 bytes in 377 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 19 00:40:48.782630 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi [None req-b31802f8-c0ee-4bc7-b9fb-9b86c54b5630 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeNegativeTest-server-1522291846", "imageRef": "b387b4f8-f75e-4a68-b49c-a015bf61c46b", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "635fade4-7ee9-45f4-8020-ceb7d6bf784a"}], "security_groups": [{"name": "tempest-securitygroup--450575057"}], "key_name": "tempest-keypair-80676479", "user_data": "}} {{(pid=125640) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 19 00:40:48.793154 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-d177f1c8-52f3-49b8-8e67-8fb866b28204 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 202 len: 402 microversion: 2.20 time: 2.728865 Oct 19 00:40:48.800904 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 8/18] 10.222.0.26 () {62 vars in 1188 bytes} [Wed Oct 19 00:40:46 2022] POST /compute/v2.1/servers => generated 402 bytes in 2737 msecs (HTTP/1.1 202) 10 headers in 453 bytes (1 switches on core 0) Oct 19 00:40:48.806166 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-386a06a1-cb22-498f-bbb2-a0395fe8d4b9 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:40:48.815582 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-386a06a1-cb22-498f-bbb2-a0395fe8d4b9 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] [instance: e8a28aad-697f-4dee-8254-6f08e9d76168] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:40:48.855816 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-386a06a1-cb22-498f-bbb2-a0395fe8d4b9 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:40:48.869912 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-386a06a1-cb22-498f-bbb2-a0395fe8d4b9 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.014s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:40:48.870374 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-386a06a1-cb22-498f-bbb2-a0395fe8d4b9 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:40:48.875347 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-386a06a1-cb22-498f-bbb2-a0395fe8d4b9 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:40:48.880436 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-386a06a1-cb22-498f-bbb2-a0395fe8d4b9 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:40:48.883153 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-386a06a1-cb22-498f-bbb2-a0395fe8d4b9 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.008s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:40:49.428512 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-386a06a1-cb22-498f-bbb2-a0395fe8d4b9 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8118f0c698aa428a93cb054abac30c2e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f249d9041c6749f58973a5b0867b07ce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:40:49.460068 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-386a06a1-cb22-498f-bbb2-a0395fe8d4b9 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] 10.222.0.26 "GET /compute/v2.1/servers/e8a28aad-697f-4dee-8254-6f08e9d76168" status: 200 len: 1210 microversion: 2.1 time: 0.657922 Oct 19 00:40:49.472105 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 9/19] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:40:48 2022] GET /compute/v2.1/servers/e8a28aad-697f-4dee-8254-6f08e9d76168 => generated 1210 bytes in 671 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:40:49.491555 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-c8c6cfa9-7ce6-4fd8-b8c0-f63e8e89256a tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:40:49.492436 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-c8c6cfa9-7ce6-4fd8-b8c0-f63e8e89256a tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] [instance: b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:40:49.503285 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.network.neutron [None req-b31802f8-c0ee-4bc7-b9fb-9b86c54b5630 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] validate_networks() for [('635fade4-7ee9-45f4-8020-ceb7d6bf784a', None, None, None, None, None)] {{(pid=125640) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 19 00:40:49.762254 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-c8c6cfa9-7ce6-4fd8-b8c0-f63e8e89256a tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:40:49.777682 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-c8c6cfa9-7ce6-4fd8-b8c0-f63e8e89256a tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:40:49.804410 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-c8c6cfa9-7ce6-4fd8-b8c0-f63e8e89256a tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:40:49.828637 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-c8c6cfa9-7ce6-4fd8-b8c0-f63e8e89256a tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] 10.222.0.26 "GET /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2" status: 200 len: 1281 microversion: 2.20 time: 0.353650 Oct 19 00:40:49.834458 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 10/20] 10.222.0.26 () {60 vars in 1277 bytes} [Wed Oct 19 00:40:49 2022] GET /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2 => generated 1281 bytes in 361 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 19 00:40:49.864909 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-42c69977-3230-4d62-b2e4-f372685d0d62 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:40:49.869424 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-42c69977-3230-4d62-b2e4-f372685d0d62 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] [instance: ab4c73d6-9203-4110-93af-4a78428c672b] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:40:49.909262 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-42c69977-3230-4d62-b2e4-f372685d0d62 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:40:49.915912 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-42c69977-3230-4d62-b2e4-f372685d0d62 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:40:49.924834 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-42c69977-3230-4d62-b2e4-f372685d0d62 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:40:49.930050 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-42c69977-3230-4d62-b2e4-f372685d0d62 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:40:49.931542 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-42c69977-3230-4d62-b2e4-f372685d0d62 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:40:49.935937 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-42c69977-3230-4d62-b2e4-f372685d0d62 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:40:50.292700 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-42c69977-3230-4d62-b2e4-f372685d0d62 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd3f0d615187d4779bdd7da8c9f24c796', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0151d58abd047a692f5cc4e7d05bea5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:40:50.305136 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-42c69977-3230-4d62-b2e4-f372685d0d62 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] 10.222.0.26 "GET /compute/v2.1/servers/ab4c73d6-9203-4110-93af-4a78428c672b" status: 200 len: 1282 microversion: 2.1 time: 0.469251 Oct 19 00:40:50.320147 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 11/21] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:40:49 2022] GET /compute/v2.1/servers/ab4c73d6-9203-4110-93af-4a78428c672b => generated 1282 bytes in 485 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:40:50.499290 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-3a4719dc-48ff-4a0c-810b-79fdfffa71e4 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:40:50.499906 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-3a4719dc-48ff-4a0c-810b-79fdfffa71e4 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] [instance: e8a28aad-697f-4dee-8254-6f08e9d76168] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:40:50.514909 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-3a4719dc-48ff-4a0c-810b-79fdfffa71e4 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:40:50.515426 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-3a4719dc-48ff-4a0c-810b-79fdfffa71e4 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:40:50.515933 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-3a4719dc-48ff-4a0c-810b-79fdfffa71e4 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:40:50.518397 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-3a4719dc-48ff-4a0c-810b-79fdfffa71e4 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:40:50.522223 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-3a4719dc-48ff-4a0c-810b-79fdfffa71e4 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:40:50.522955 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-3a4719dc-48ff-4a0c-810b-79fdfffa71e4 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:40:50.718598 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-3a4719dc-48ff-4a0c-810b-79fdfffa71e4 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8118f0c698aa428a93cb054abac30c2e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f249d9041c6749f58973a5b0867b07ce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:40:50.730263 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-3a4719dc-48ff-4a0c-810b-79fdfffa71e4 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] 10.222.0.26 "GET /compute/v2.1/servers/e8a28aad-697f-4dee-8254-6f08e9d76168" status: 200 len: 1266 microversion: 2.1 time: 0.234948 Oct 19 00:40:50.735850 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 12/22] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:40:50 2022] GET /compute/v2.1/servers/e8a28aad-697f-4dee-8254-6f08e9d76168 => generated 1266 bytes in 238 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:40:50.863119 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-5b575f98-33e1-42fa-b71c-c1715003d57d tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:40:50.863796 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-5b575f98-33e1-42fa-b71c-c1715003d57d tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] [instance: b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:40:50.878631 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-5b575f98-33e1-42fa-b71c-c1715003d57d tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:40:50.879219 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-5b575f98-33e1-42fa-b71c-c1715003d57d tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:40:50.883858 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-5b575f98-33e1-42fa-b71c-c1715003d57d tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:40:50.888406 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-5b575f98-33e1-42fa-b71c-c1715003d57d tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:40:50.889439 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-5b575f98-33e1-42fa-b71c-c1715003d57d tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:40:50.890884 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-5b575f98-33e1-42fa-b71c-c1715003d57d tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:40:51.326002 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-5b575f98-33e1-42fa-b71c-c1715003d57d tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:40:51.354271 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-5b575f98-33e1-42fa-b71c-c1715003d57d tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:40:51.358907 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-5b575f98-33e1-42fa-b71c-c1715003d57d tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:40:51.360081 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-5b575f98-33e1-42fa-b71c-c1715003d57d tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] 10.222.0.26 "GET /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2" status: 200 len: 1277 microversion: 2.20 time: 0.503083 Oct 19 00:40:51.374226 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 13/23] 10.222.0.26 () {60 vars in 1277 bytes} [Wed Oct 19 00:40:50 2022] GET /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2 => generated 1277 bytes in 517 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 19 00:40:51.401763 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-cf00484e-56df-4ad3-8b1a-fb4f0dae2f8d tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:40:51.403964 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-cf00484e-56df-4ad3-8b1a-fb4f0dae2f8d tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] [instance: ab4c73d6-9203-4110-93af-4a78428c672b] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:40:51.436445 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-cf00484e-56df-4ad3-8b1a-fb4f0dae2f8d tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:40:51.440150 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-cf00484e-56df-4ad3-8b1a-fb4f0dae2f8d tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:40:51.443202 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-cf00484e-56df-4ad3-8b1a-fb4f0dae2f8d tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:40:51.447072 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-cf00484e-56df-4ad3-8b1a-fb4f0dae2f8d tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:40:51.449527 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-cf00484e-56df-4ad3-8b1a-fb4f0dae2f8d tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:40:51.450850 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-cf00484e-56df-4ad3-8b1a-fb4f0dae2f8d tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:40:51.678283 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.virt.hardware [None req-b31802f8-c0ee-4bc7-b9fb-9b86c54b5630 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Flavor limits 0:0:0 {{(pid=125640) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 19 00:40:51.679606 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.virt.hardware [None req-b31802f8-c0ee-4bc7-b9fb-9b86c54b5630 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Image limits 0:0:0 {{(pid=125640) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 19 00:40:51.680312 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.virt.hardware [None req-b31802f8-c0ee-4bc7-b9fb-9b86c54b5630 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Flavor pref 0:0:0 {{(pid=125640) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 19 00:40:51.680946 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.virt.hardware [None req-b31802f8-c0ee-4bc7-b9fb-9b86c54b5630 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Image pref 0:0:0 {{(pid=125640) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 19 00:40:51.681595 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.virt.hardware [None req-b31802f8-c0ee-4bc7-b9fb-9b86c54b5630 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=125640) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 19 00:40:51.714299 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-b31802f8-c0ee-4bc7-b9fb-9b86c54b5630 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:40:51.715896 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-b31802f8-c0ee-4bc7-b9fb-9b86c54b5630 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:40:51.718632 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-b31802f8-c0ee-4bc7-b9fb-9b86c54b5630 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:40:51.721755 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-b31802f8-c0ee-4bc7-b9fb-9b86c54b5630 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:40:51.728150 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-b31802f8-c0ee-4bc7-b9fb-9b86c54b5630 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:40:51.731352 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-b31802f8-c0ee-4bc7-b9fb-9b86c54b5630 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:40:51.850316 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.quota [None req-b31802f8-c0ee-4bc7-b9fb-9b86c54b5630 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Getting quotas for project 0b35c2038f6b4d0a9012b004f1416c00. Resources: {'cores', 'ram', 'instances'} {{(pid=125640) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 19 00:40:51.882861 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.quota [None req-b31802f8-c0ee-4bc7-b9fb-9b86c54b5630 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Getting quotas for user 6f8725c4d05b4dd79b05b43fba15980b and project 0b35c2038f6b4d0a9012b004f1416c00. Resources: {'cores', 'ram', 'instances'} {{(pid=125640) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 19 00:40:51.947001 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.compute.api [None req-b31802f8-c0ee-4bc7-b9fb-9b86c54b5630 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Going to run 1 instances... {{(pid=125640) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 19 00:40:51.960976 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.compute.api [None req-b31802f8-c0ee-4bc7-b9fb-9b86c54b5630 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] [instance: dc8b8a6e-eab7-4f45-8f5d-916c52eec398] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='b387b4f8-f75e-4a68-b49c-a015bf61c46b',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=125640) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 19 00:40:52.169409 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: INFO nova.api.openstack.requestlog [None req-b31802f8-c0ee-4bc7-b9fb-9b86c54b5630 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 202 len: 401 microversion: 2.1 time: 3.398541 Oct 19 00:40:52.175442 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: [pid: 125640|app: 0|req: 11/24] 10.222.0.26 () {60 vars in 1147 bytes} [Wed Oct 19 00:40:48 2022] POST /compute/v2.1/servers => generated 401 bytes in 3409 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 19 00:40:52.194257 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi [None req-dddbb269-cec4-4208-ac59-c76fbaef32ad tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Calling method '>' {{(pid=125640) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:40:52.197215 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.compute.api [None req-dddbb269-cec4-4208-ac59-c76fbaef32ad tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] [instance: e8a28aad-697f-4dee-8254-6f08e9d76168] Fetching instance by UUID {{(pid=125640) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:40:52.221153 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-dddbb269-cec4-4208-ac59-c76fbaef32ad tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:40:52.221865 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-dddbb269-cec4-4208-ac59-c76fbaef32ad tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:40:52.228566 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-dddbb269-cec4-4208-ac59-c76fbaef32ad tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:40:52.232154 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-dddbb269-cec4-4208-ac59-c76fbaef32ad tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:40:52.235565 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-dddbb269-cec4-4208-ac59-c76fbaef32ad tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:40:52.237561 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-dddbb269-cec4-4208-ac59-c76fbaef32ad tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:40:52.651728 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.policy [None req-dddbb269-cec4-4208-ac59-c76fbaef32ad tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8118f0c698aa428a93cb054abac30c2e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f249d9041c6749f58973a5b0867b07ce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125640) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:40:52.663186 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: INFO nova.api.openstack.requestlog [None req-dddbb269-cec4-4208-ac59-c76fbaef32ad tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] 10.222.0.26 "GET /compute/v2.1/servers/e8a28aad-697f-4dee-8254-6f08e9d76168" status: 200 len: 1272 microversion: 2.1 time: 0.485717 Oct 19 00:40:52.663805 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: [pid: 125640|app: 0|req: 12/25] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:40:52 2022] GET /compute/v2.1/servers/e8a28aad-697f-4dee-8254-6f08e9d76168 => generated 1272 bytes in 488 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:40:52.668465 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-cf00484e-56df-4ad3-8b1a-fb4f0dae2f8d tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd3f0d615187d4779bdd7da8c9f24c796', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0151d58abd047a692f5cc4e7d05bea5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:40:52.707378 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-cf00484e-56df-4ad3-8b1a-fb4f0dae2f8d tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] 10.222.0.26 "GET /compute/v2.1/servers/ab4c73d6-9203-4110-93af-4a78428c672b" status: 200 len: 1280 microversion: 2.1 time: 1.327863 Oct 19 00:40:52.707974 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 14/26] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:40:51 2022] GET /compute/v2.1/servers/ab4c73d6-9203-4110-93af-4a78428c672b => generated 1280 bytes in 1333 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:40:52.710088 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi [None req-5826e170-fd6a-4923-8998-f924697fd6fb tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Calling method '>' {{(pid=125640) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:40:52.711774 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.compute.api [None req-5826e170-fd6a-4923-8998-f924697fd6fb tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] [instance: dc8b8a6e-eab7-4f45-8f5d-916c52eec398] Fetching instance by UUID {{(pid=125640) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:40:52.747871 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-da1557e8-f14a-438a-8434-b2ce9b135405 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:40:52.748268 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-da1557e8-f14a-438a-8434-b2ce9b135405 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] [instance: b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:40:52.795206 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-da1557e8-f14a-438a-8434-b2ce9b135405 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:40:52.797858 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-da1557e8-f14a-438a-8434-b2ce9b135405 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:40:52.798351 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-da1557e8-f14a-438a-8434-b2ce9b135405 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:40:52.800189 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-da1557e8-f14a-438a-8434-b2ce9b135405 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:40:52.806336 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-da1557e8-f14a-438a-8434-b2ce9b135405 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:40:52.809091 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-da1557e8-f14a-438a-8434-b2ce9b135405 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:40:53.024592 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.policy [None req-5826e170-fd6a-4923-8998-f924697fd6fb tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f8725c4d05b4dd79b05b43fba15980b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0b35c2038f6b4d0a9012b004f1416c00', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125640) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:40:53.032575 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: INFO nova.api.openstack.requestlog [None req-5826e170-fd6a-4923-8998-f924697fd6fb tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] 10.222.0.26 "GET /compute/v2.1/servers/dc8b8a6e-eab7-4f45-8f5d-916c52eec398" status: 200 len: 1240 microversion: 2.1 time: 0.348613 Oct 19 00:40:53.039106 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: [pid: 125640|app: 0|req: 13/27] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:40:52 2022] GET /compute/v2.1/servers/dc8b8a6e-eab7-4f45-8f5d-916c52eec398 => generated 1240 bytes in 373 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:40:53.695899 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi [None req-da0f0d3b-b128-4bdd-8ce6-15f0002c1439 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Calling method '>' {{(pid=125640) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:40:53.696766 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.compute.api [None req-da0f0d3b-b128-4bdd-8ce6-15f0002c1439 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] [instance: e8a28aad-697f-4dee-8254-6f08e9d76168] Fetching instance by UUID {{(pid=125640) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:40:53.712458 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-da0f0d3b-b128-4bdd-8ce6-15f0002c1439 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:40:53.717973 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-da0f0d3b-b128-4bdd-8ce6-15f0002c1439 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:40:53.718685 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-da0f0d3b-b128-4bdd-8ce6-15f0002c1439 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:40:53.719659 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-da0f0d3b-b128-4bdd-8ce6-15f0002c1439 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:40:53.724732 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-da0f0d3b-b128-4bdd-8ce6-15f0002c1439 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:40:53.725865 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-da0f0d3b-b128-4bdd-8ce6-15f0002c1439 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:40:53.887866 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-da1557e8-f14a-438a-8434-b2ce9b135405 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:40:53.901197 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-da1557e8-f14a-438a-8434-b2ce9b135405 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:40:53.906396 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-da1557e8-f14a-438a-8434-b2ce9b135405 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:40:53.918338 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-da1557e8-f14a-438a-8434-b2ce9b135405 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] 10.222.0.26 "GET /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2" status: 200 len: 1333 microversion: 2.20 time: 1.209255 Oct 19 00:40:53.937801 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 15/28] 10.222.0.26 () {60 vars in 1277 bytes} [Wed Oct 19 00:40:52 2022] GET /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2 => generated 1333 bytes in 1228 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 19 00:40:53.965353 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-79f099df-f5ae-49be-a909-25987d3f5317 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:40:53.983221 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-79f099df-f5ae-49be-a909-25987d3f5317 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] [instance: ab4c73d6-9203-4110-93af-4a78428c672b] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:40:53.995923 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-79f099df-f5ae-49be-a909-25987d3f5317 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:40:54.017252 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-79f099df-f5ae-49be-a909-25987d3f5317 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.021s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:40:54.020688 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-79f099df-f5ae-49be-a909-25987d3f5317 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:40:54.021689 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-79f099df-f5ae-49be-a909-25987d3f5317 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:40:54.027153 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-79f099df-f5ae-49be-a909-25987d3f5317 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:40:54.035289 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-79f099df-f5ae-49be-a909-25987d3f5317 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.008s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:40:54.114291 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.policy [None req-da0f0d3b-b128-4bdd-8ce6-15f0002c1439 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8118f0c698aa428a93cb054abac30c2e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f249d9041c6749f58973a5b0867b07ce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125640) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:40:54.141275 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: INFO nova.api.openstack.requestlog [None req-da0f0d3b-b128-4bdd-8ce6-15f0002c1439 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] 10.222.0.26 "GET /compute/v2.1/servers/e8a28aad-697f-4dee-8254-6f08e9d76168" status: 200 len: 1272 microversion: 2.1 time: 0.442050 Oct 19 00:40:54.141935 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: [pid: 125640|app: 0|req: 14/29] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:40:53 2022] GET /compute/v2.1/servers/e8a28aad-697f-4dee-8254-6f08e9d76168 => generated 1272 bytes in 452 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:40:54.164705 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi [None req-5a403d30-0de7-451b-8620-479d6a05354a tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Calling method '>' {{(pid=125640) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:40:54.177656 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.compute.api [None req-5a403d30-0de7-451b-8620-479d6a05354a tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] [instance: dc8b8a6e-eab7-4f45-8f5d-916c52eec398] Fetching instance by UUID {{(pid=125640) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:40:54.222021 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-5a403d30-0de7-451b-8620-479d6a05354a tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:40:54.222599 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-5a403d30-0de7-451b-8620-479d6a05354a tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:40:54.224593 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-5a403d30-0de7-451b-8620-479d6a05354a tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:40:54.231539 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-5a403d30-0de7-451b-8620-479d6a05354a tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:40:54.242431 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-5a403d30-0de7-451b-8620-479d6a05354a tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:40:54.243003 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-5a403d30-0de7-451b-8620-479d6a05354a tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.011s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:40:54.320773 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-79f099df-f5ae-49be-a909-25987d3f5317 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd3f0d615187d4779bdd7da8c9f24c796', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0151d58abd047a692f5cc4e7d05bea5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:40:54.338437 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-79f099df-f5ae-49be-a909-25987d3f5317 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] 10.222.0.26 "GET /compute/v2.1/servers/ab4c73d6-9203-4110-93af-4a78428c672b" status: 200 len: 1280 microversion: 2.1 time: 0.397848 Oct 19 00:40:54.340580 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 16/30] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:40:53 2022] GET /compute/v2.1/servers/ab4c73d6-9203-4110-93af-4a78428c672b => generated 1280 bytes in 401 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:40:54.466288 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.policy [None req-5a403d30-0de7-451b-8620-479d6a05354a tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f8725c4d05b4dd79b05b43fba15980b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0b35c2038f6b4d0a9012b004f1416c00', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125640) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:40:54.489856 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: INFO nova.api.openstack.requestlog [None req-5a403d30-0de7-451b-8620-479d6a05354a tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] 10.222.0.26 "GET /compute/v2.1/servers/dc8b8a6e-eab7-4f45-8f5d-916c52eec398" status: 200 len: 1236 microversion: 2.1 time: 0.334817 Oct 19 00:40:54.494260 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: [pid: 125640|app: 0|req: 15/31] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:40:54 2022] GET /compute/v2.1/servers/dc8b8a6e-eab7-4f45-8f5d-916c52eec398 => generated 1236 bytes in 345 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:40:54.954745 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-59cf6c26-910a-4745-9f31-71ee323c113e tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:40:54.956368 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-59cf6c26-910a-4745-9f31-71ee323c113e tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] [instance: b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:40:54.997063 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-59cf6c26-910a-4745-9f31-71ee323c113e tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:40:55.011502 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-59cf6c26-910a-4745-9f31-71ee323c113e tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:40:55.014894 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-59cf6c26-910a-4745-9f31-71ee323c113e tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.014s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:40:55.023796 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-59cf6c26-910a-4745-9f31-71ee323c113e tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:40:55.028850 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-59cf6c26-910a-4745-9f31-71ee323c113e tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:40:55.035221 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-59cf6c26-910a-4745-9f31-71ee323c113e tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:40:55.166563 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi [None req-8e362a8f-54c3-4fd1-a3c0-9a2988f04903 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Calling method '>' {{(pid=125640) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:40:55.167540 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.compute.api [None req-8e362a8f-54c3-4fd1-a3c0-9a2988f04903 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] [instance: e8a28aad-697f-4dee-8254-6f08e9d76168] Fetching instance by UUID {{(pid=125640) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:40:55.175932 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-8e362a8f-54c3-4fd1-a3c0-9a2988f04903 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:40:55.180629 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-8e362a8f-54c3-4fd1-a3c0-9a2988f04903 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:40:55.187358 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-8e362a8f-54c3-4fd1-a3c0-9a2988f04903 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:40:55.188410 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-8e362a8f-54c3-4fd1-a3c0-9a2988f04903 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:40:55.189179 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-8e362a8f-54c3-4fd1-a3c0-9a2988f04903 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:40:55.192671 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-8e362a8f-54c3-4fd1-a3c0-9a2988f04903 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:40:55.333515 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-59cf6c26-910a-4745-9f31-71ee323c113e tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:40:55.347849 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-59cf6c26-910a-4745-9f31-71ee323c113e tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:40:55.353771 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.policy [None req-8e362a8f-54c3-4fd1-a3c0-9a2988f04903 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8118f0c698aa428a93cb054abac30c2e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f249d9041c6749f58973a5b0867b07ce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125640) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:40:55.359092 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-59cf6c26-910a-4745-9f31-71ee323c113e tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:40:55.360273 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-59cf6c26-910a-4745-9f31-71ee323c113e tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] 10.222.0.26 "GET /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2" status: 200 len: 1339 microversion: 2.20 time: 0.410819 Oct 19 00:40:55.360841 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 17/32] 10.222.0.26 () {60 vars in 1277 bytes} [Wed Oct 19 00:40:54 2022] GET /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2 => generated 1339 bytes in 412 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 19 00:40:55.369269 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: INFO nova.api.openstack.requestlog [None req-8e362a8f-54c3-4fd1-a3c0-9a2988f04903 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] 10.222.0.26 "GET /compute/v2.1/servers/e8a28aad-697f-4dee-8254-6f08e9d76168" status: 200 len: 1272 microversion: 2.1 time: 0.203390 Oct 19 00:40:55.379134 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-c42e8168-0c57-47c9-859e-26cf1b3c0526 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:40:55.380130 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: [pid: 125640|app: 0|req: 16/33] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:40:55 2022] GET /compute/v2.1/servers/e8a28aad-697f-4dee-8254-6f08e9d76168 => generated 1272 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:40:55.403317 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-c42e8168-0c57-47c9-859e-26cf1b3c0526 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] [instance: ab4c73d6-9203-4110-93af-4a78428c672b] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:40:55.430185 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-c42e8168-0c57-47c9-859e-26cf1b3c0526 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:40:55.435672 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-c42e8168-0c57-47c9-859e-26cf1b3c0526 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:40:55.438232 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-c42e8168-0c57-47c9-859e-26cf1b3c0526 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:40:55.442647 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-c42e8168-0c57-47c9-859e-26cf1b3c0526 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:40:55.444622 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-c42e8168-0c57-47c9-859e-26cf1b3c0526 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:40:55.456195 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-c42e8168-0c57-47c9-859e-26cf1b3c0526 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:40:55.527479 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi [None req-6189560f-38ff-4643-b095-e3d6f72b6d93 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Calling method '>' {{(pid=125640) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:40:55.528630 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.compute.api [None req-6189560f-38ff-4643-b095-e3d6f72b6d93 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] [instance: dc8b8a6e-eab7-4f45-8f5d-916c52eec398] Fetching instance by UUID {{(pid=125640) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:40:55.543313 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-6189560f-38ff-4643-b095-e3d6f72b6d93 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:40:55.550283 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-6189560f-38ff-4643-b095-e3d6f72b6d93 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:40:55.551301 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-6189560f-38ff-4643-b095-e3d6f72b6d93 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:40:55.553672 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-6189560f-38ff-4643-b095-e3d6f72b6d93 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:40:55.566994 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-6189560f-38ff-4643-b095-e3d6f72b6d93 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.013s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:40:55.567461 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-6189560f-38ff-4643-b095-e3d6f72b6d93 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:40:55.761862 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-c42e8168-0c57-47c9-859e-26cf1b3c0526 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd3f0d615187d4779bdd7da8c9f24c796', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0151d58abd047a692f5cc4e7d05bea5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:40:55.770661 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-c42e8168-0c57-47c9-859e-26cf1b3c0526 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] 10.222.0.26 "GET /compute/v2.1/servers/ab4c73d6-9203-4110-93af-4a78428c672b" status: 200 len: 1280 microversion: 2.1 time: 0.396921 Oct 19 00:40:55.782525 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 18/34] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:40:55 2022] GET /compute/v2.1/servers/ab4c73d6-9203-4110-93af-4a78428c672b => generated 1280 bytes in 416 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 19 00:40:56.019438 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.policy [None req-6189560f-38ff-4643-b095-e3d6f72b6d93 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f8725c4d05b4dd79b05b43fba15980b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0b35c2038f6b4d0a9012b004f1416c00', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125640) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:40:56.081357 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: INFO nova.api.openstack.requestlog [None req-6189560f-38ff-4643-b095-e3d6f72b6d93 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] 10.222.0.26 "GET /compute/v2.1/servers/dc8b8a6e-eab7-4f45-8f5d-916c52eec398" status: 200 len: 1292 microversion: 2.1 time: 0.562536 Oct 19 00:40:56.084314 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: [pid: 125640|app: 0|req: 17/35] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:40:55 2022] GET /compute/v2.1/servers/dc8b8a6e-eab7-4f45-8f5d-916c52eec398 => generated 1292 bytes in 571 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:40:56.396667 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi [None req-738ef5d6-81c6-4449-9d11-a7d9b3917ec2 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Calling method '>' {{(pid=125640) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:40:56.399510 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-c42104ff-f732-436c-b3f4-519fac989908 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:40:56.400780 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.compute.api [None req-738ef5d6-81c6-4449-9d11-a7d9b3917ec2 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] [instance: e8a28aad-697f-4dee-8254-6f08e9d76168] Fetching instance by UUID {{(pid=125640) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:40:56.401856 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-c42104ff-f732-436c-b3f4-519fac989908 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] [instance: b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:40:56.421381 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-c42104ff-f732-436c-b3f4-519fac989908 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:40:56.429658 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-738ef5d6-81c6-4449-9d11-a7d9b3917ec2 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:40:56.433263 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-c42104ff-f732-436c-b3f4-519fac989908 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:40:56.433805 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-738ef5d6-81c6-4449-9d11-a7d9b3917ec2 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:40:56.436150 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-c42104ff-f732-436c-b3f4-519fac989908 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.013s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:40:56.438087 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-738ef5d6-81c6-4449-9d11-a7d9b3917ec2 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:40:56.442562 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-c42104ff-f732-436c-b3f4-519fac989908 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:40:56.443890 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-738ef5d6-81c6-4449-9d11-a7d9b3917ec2 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:40:56.446414 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-c42104ff-f732-436c-b3f4-519fac989908 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:40:56.446916 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-c42104ff-f732-436c-b3f4-519fac989908 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:40:56.447651 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-738ef5d6-81c6-4449-9d11-a7d9b3917ec2 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:40:56.450356 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-738ef5d6-81c6-4449-9d11-a7d9b3917ec2 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:40:56.967067 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-c42104ff-f732-436c-b3f4-519fac989908 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:40:56.978102 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-c42104ff-f732-436c-b3f4-519fac989908 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:40:56.983037 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-c42104ff-f732-436c-b3f4-519fac989908 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:40:56.986469 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-c42104ff-f732-436c-b3f4-519fac989908 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] 10.222.0.26 "GET /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2" status: 200 len: 1339 microversion: 2.20 time: 0.597975 Oct 19 00:40:57.010609 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 19/36] 10.222.0.26 () {60 vars in 1277 bytes} [Wed Oct 19 00:40:56 2022] GET /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2 => generated 1339 bytes in 623 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 19 00:40:57.015650 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-5ac5d80e-73b8-44d8-9480-b293740a0113 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:40:57.031337 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-5ac5d80e-73b8-44d8-9480-b293740a0113 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] [instance: ab4c73d6-9203-4110-93af-4a78428c672b] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:40:57.054383 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-5ac5d80e-73b8-44d8-9480-b293740a0113 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:40:57.061059 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-5ac5d80e-73b8-44d8-9480-b293740a0113 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:40:57.061859 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-5ac5d80e-73b8-44d8-9480-b293740a0113 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:40:57.062922 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-5ac5d80e-73b8-44d8-9480-b293740a0113 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:40:57.069756 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-5ac5d80e-73b8-44d8-9480-b293740a0113 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:40:57.070447 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-5ac5d80e-73b8-44d8-9480-b293740a0113 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:40:57.529420 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.policy [None req-738ef5d6-81c6-4449-9d11-a7d9b3917ec2 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8118f0c698aa428a93cb054abac30c2e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f249d9041c6749f58973a5b0867b07ce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125640) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:40:57.542196 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: INFO nova.api.openstack.requestlog [None req-738ef5d6-81c6-4449-9d11-a7d9b3917ec2 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] 10.222.0.26 "GET /compute/v2.1/servers/e8a28aad-697f-4dee-8254-6f08e9d76168" status: 200 len: 1272 microversion: 2.1 time: 1.148795 Oct 19 00:40:57.559339 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: [pid: 125640|app: 0|req: 18/37] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:40:56 2022] GET /compute/v2.1/servers/e8a28aad-697f-4dee-8254-6f08e9d76168 => generated 1272 bytes in 1167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 19 00:40:57.573057 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi [None req-adb91242-2286-44a2-812d-d0a40f1d50c3 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Calling method '>' {{(pid=125640) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:40:57.575820 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.compute.api [None req-adb91242-2286-44a2-812d-d0a40f1d50c3 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] [instance: dc8b8a6e-eab7-4f45-8f5d-916c52eec398] Fetching instance by UUID {{(pid=125640) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:40:57.595383 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-adb91242-2286-44a2-812d-d0a40f1d50c3 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:40:57.597561 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-adb91242-2286-44a2-812d-d0a40f1d50c3 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:40:57.598211 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-adb91242-2286-44a2-812d-d0a40f1d50c3 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:40:57.599793 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-adb91242-2286-44a2-812d-d0a40f1d50c3 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:40:57.602537 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-adb91242-2286-44a2-812d-d0a40f1d50c3 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:40:57.603203 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-adb91242-2286-44a2-812d-d0a40f1d50c3 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:40:57.901062 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-5ac5d80e-73b8-44d8-9480-b293740a0113 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd3f0d615187d4779bdd7da8c9f24c796', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0151d58abd047a692f5cc4e7d05bea5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:40:57.910034 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-5ac5d80e-73b8-44d8-9480-b293740a0113 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] 10.222.0.26 "GET /compute/v2.1/servers/ab4c73d6-9203-4110-93af-4a78428c672b" status: 200 len: 1322 microversion: 2.1 time: 0.897834 Oct 19 00:40:57.915828 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 20/38] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:40:57 2022] GET /compute/v2.1/servers/ab4c73d6-9203-4110-93af-4a78428c672b => generated 1322 bytes in 905 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:40:58.023660 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-46a7edaf-9ebe-4910-b7ae-810cd341aa42 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:40:58.024702 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-46a7edaf-9ebe-4910-b7ae-810cd341aa42 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] [instance: b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:40:58.035622 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-46a7edaf-9ebe-4910-b7ae-810cd341aa42 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:40:58.047281 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-46a7edaf-9ebe-4910-b7ae-810cd341aa42 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.012s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:40:58.048089 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-46a7edaf-9ebe-4910-b7ae-810cd341aa42 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:40:58.049834 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-46a7edaf-9ebe-4910-b7ae-810cd341aa42 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:40:58.050564 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-46a7edaf-9ebe-4910-b7ae-810cd341aa42 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:40:58.051223 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-46a7edaf-9ebe-4910-b7ae-810cd341aa42 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:40:58.141232 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.policy [None req-adb91242-2286-44a2-812d-d0a40f1d50c3 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f8725c4d05b4dd79b05b43fba15980b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0b35c2038f6b4d0a9012b004f1416c00', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125640) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:40:58.172474 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: INFO nova.api.openstack.requestlog [None req-adb91242-2286-44a2-812d-d0a40f1d50c3 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] 10.222.0.26 "GET /compute/v2.1/servers/dc8b8a6e-eab7-4f45-8f5d-916c52eec398" status: 200 len: 1298 microversion: 2.1 time: 0.608984 Oct 19 00:40:58.176625 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: [pid: 125640|app: 0|req: 19/39] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:40:57 2022] GET /compute/v2.1/servers/dc8b8a6e-eab7-4f45-8f5d-916c52eec398 => generated 1298 bytes in 617 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:40:58.570838 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi [None req-4264afb3-a30e-4ccf-8f9b-c1edc4e9add3 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Calling method '>' {{(pid=125640) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:40:58.571722 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.compute.api [None req-4264afb3-a30e-4ccf-8f9b-c1edc4e9add3 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] [instance: e8a28aad-697f-4dee-8254-6f08e9d76168] Fetching instance by UUID {{(pid=125640) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:40:58.583272 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-4264afb3-a30e-4ccf-8f9b-c1edc4e9add3 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:40:58.584031 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-4264afb3-a30e-4ccf-8f9b-c1edc4e9add3 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:40:58.585769 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-4264afb3-a30e-4ccf-8f9b-c1edc4e9add3 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:40:58.586653 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG dbcounter [-] [125641] Writing DB stats nova_cell0:SELECT=6 {{(pid=125641) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 00:40:58.588136 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-4264afb3-a30e-4ccf-8f9b-c1edc4e9add3 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:40:58.595345 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-4264afb3-a30e-4ccf-8f9b-c1edc4e9add3 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:40:58.600198 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-4264afb3-a30e-4ccf-8f9b-c1edc4e9add3 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:40:59.507204 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-46a7edaf-9ebe-4910-b7ae-810cd341aa42 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:40:59.516293 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-46a7edaf-9ebe-4910-b7ae-810cd341aa42 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:40:59.541139 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-46a7edaf-9ebe-4910-b7ae-810cd341aa42 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:40:59.553280 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-46a7edaf-9ebe-4910-b7ae-810cd341aa42 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] 10.222.0.26 "GET /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2" status: 200 len: 1407 microversion: 2.20 time: 1.534475 Oct 19 00:40:59.572222 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 21/40] 10.222.0.26 () {60 vars in 1277 bytes} [Wed Oct 19 00:40:58 2022] GET /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2 => generated 1407 bytes in 1559 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 19 00:40:59.600197 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-ea1aabe8-218b-414d-9aa7-eb925790ac08 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:40:59.605002 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-ea1aabe8-218b-414d-9aa7-eb925790ac08 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] [instance: ab4c73d6-9203-4110-93af-4a78428c672b] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:40:59.632558 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ea1aabe8-218b-414d-9aa7-eb925790ac08 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:40:59.642181 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ea1aabe8-218b-414d-9aa7-eb925790ac08 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:40:59.651371 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ea1aabe8-218b-414d-9aa7-eb925790ac08 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.013s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:40:59.653773 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ea1aabe8-218b-414d-9aa7-eb925790ac08 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:40:59.657134 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ea1aabe8-218b-414d-9aa7-eb925790ac08 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:40:59.659131 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ea1aabe8-218b-414d-9aa7-eb925790ac08 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:40:59.781555 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG dbcounter [-] [125641] Writing DB stats nova_cell0:SELECT=22 {{(pid=125641) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 00:41:00.361095 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.policy [None req-4264afb3-a30e-4ccf-8f9b-c1edc4e9add3 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8118f0c698aa428a93cb054abac30c2e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f249d9041c6749f58973a5b0867b07ce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125640) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:41:00.381883 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: INFO nova.api.openstack.requestlog [None req-4264afb3-a30e-4ccf-8f9b-c1edc4e9add3 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] 10.222.0.26 "GET /compute/v2.1/servers/e8a28aad-697f-4dee-8254-6f08e9d76168" status: 200 len: 1314 microversion: 2.1 time: 1.812004 Oct 19 00:41:00.382573 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: [pid: 125640|app: 0|req: 20/41] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:40:58 2022] GET /compute/v2.1/servers/e8a28aad-697f-4dee-8254-6f08e9d76168 => generated 1314 bytes in 1817 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 19 00:41:00.401879 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi [None req-abb893db-b3f1-446c-bf99-7d7b92d6d118 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Calling method '>' {{(pid=125640) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:41:00.403156 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.compute.api [None req-abb893db-b3f1-446c-bf99-7d7b92d6d118 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] [instance: dc8b8a6e-eab7-4f45-8f5d-916c52eec398] Fetching instance by UUID {{(pid=125640) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:41:00.411530 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-abb893db-b3f1-446c-bf99-7d7b92d6d118 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:41:00.412540 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-abb893db-b3f1-446c-bf99-7d7b92d6d118 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:41:00.413254 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-abb893db-b3f1-446c-bf99-7d7b92d6d118 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:41:00.414859 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-abb893db-b3f1-446c-bf99-7d7b92d6d118 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:41:00.415551 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-abb893db-b3f1-446c-bf99-7d7b92d6d118 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:41:00.416251 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-abb893db-b3f1-446c-bf99-7d7b92d6d118 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:41:00.572007 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.policy [None req-abb893db-b3f1-446c-bf99-7d7b92d6d118 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f8725c4d05b4dd79b05b43fba15980b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0b35c2038f6b4d0a9012b004f1416c00', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125640) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:41:00.583154 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: INFO nova.api.openstack.requestlog [None req-abb893db-b3f1-446c-bf99-7d7b92d6d118 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] 10.222.0.26 "GET /compute/v2.1/servers/dc8b8a6e-eab7-4f45-8f5d-916c52eec398" status: 200 len: 1298 microversion: 2.1 time: 0.197257 Oct 19 00:41:00.583703 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: [pid: 125640|app: 0|req: 21/42] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:41:00 2022] GET /compute/v2.1/servers/dc8b8a6e-eab7-4f45-8f5d-916c52eec398 => generated 1298 bytes in 200 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:41:00.598881 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi [None req-84605727-22a1-4dba-9c2e-1984d6e2d162 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Calling method '>' {{(pid=125640) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:41:00.599831 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.compute.api [None req-84605727-22a1-4dba-9c2e-1984d6e2d162 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] [instance: b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2] Fetching instance by UUID {{(pid=125640) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:41:00.625156 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-84605727-22a1-4dba-9c2e-1984d6e2d162 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:41:00.625637 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-84605727-22a1-4dba-9c2e-1984d6e2d162 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:41:00.626082 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-84605727-22a1-4dba-9c2e-1984d6e2d162 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:41:00.628479 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-84605727-22a1-4dba-9c2e-1984d6e2d162 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:41:00.628479 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-84605727-22a1-4dba-9c2e-1984d6e2d162 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:41:00.629697 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-84605727-22a1-4dba-9c2e-1984d6e2d162 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:41:00.744235 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-ea1aabe8-218b-414d-9aa7-eb925790ac08 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd3f0d615187d4779bdd7da8c9f24c796', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0151d58abd047a692f5cc4e7d05bea5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:41:00.760520 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-ea1aabe8-218b-414d-9aa7-eb925790ac08 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] 10.222.0.26 "GET /compute/v2.1/servers/ab4c73d6-9203-4110-93af-4a78428c672b" status: 200 len: 1322 microversion: 2.1 time: 1.177927 Oct 19 00:41:00.764926 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 22/43] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:40:59 2022] GET /compute/v2.1/servers/ab4c73d6-9203-4110-93af-4a78428c672b => generated 1322 bytes in 1192 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:41:01.406617 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-fb2b24e6-77cb-44ab-826b-fc3423a9e09d tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:41:01.407744 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-fb2b24e6-77cb-44ab-826b-fc3423a9e09d tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] [instance: e8a28aad-697f-4dee-8254-6f08e9d76168] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:41:01.424407 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-fb2b24e6-77cb-44ab-826b-fc3423a9e09d tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:41:01.425003 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-fb2b24e6-77cb-44ab-826b-fc3423a9e09d tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:41:01.425326 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-fb2b24e6-77cb-44ab-826b-fc3423a9e09d tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:41:01.426040 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-fb2b24e6-77cb-44ab-826b-fc3423a9e09d tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:41:01.426537 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-fb2b24e6-77cb-44ab-826b-fc3423a9e09d tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:41:01.440317 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-fb2b24e6-77cb-44ab-826b-fc3423a9e09d tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:41:01.810975 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG dbcounter [-] [125640] Writing DB stats nova_cell0:SELECT=4 {{(pid=125640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 00:41:02.070089 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.policy [None req-84605727-22a1-4dba-9c2e-1984d6e2d162 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125640) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:41:02.088612 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.policy [None req-84605727-22a1-4dba-9c2e-1984d6e2d162 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125640) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:41:02.090542 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.policy [None req-84605727-22a1-4dba-9c2e-1984d6e2d162 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125640) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:41:02.098918 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: INFO nova.api.openstack.requestlog [None req-84605727-22a1-4dba-9c2e-1984d6e2d162 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] 10.222.0.26 "GET /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2" status: 200 len: 1407 microversion: 2.20 time: 1.511190 Oct 19 00:41:02.106167 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: [pid: 125640|app: 0|req: 22/44] 10.222.0.26 () {60 vars in 1277 bytes} [Wed Oct 19 00:41:00 2022] GET /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2 => generated 1407 bytes in 1519 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 19 00:41:02.117786 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi [None req-6ddd31da-551b-46df-9003-c2eb21d275a2 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Calling method '>' {{(pid=125640) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:41:02.120275 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.compute.api [None req-6ddd31da-551b-46df-9003-c2eb21d275a2 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] [instance: dc8b8a6e-eab7-4f45-8f5d-916c52eec398] Fetching instance by UUID {{(pid=125640) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:41:02.143596 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-6ddd31da-551b-46df-9003-c2eb21d275a2 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:41:02.144007 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-6ddd31da-551b-46df-9003-c2eb21d275a2 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:41:02.146427 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-6ddd31da-551b-46df-9003-c2eb21d275a2 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:41:02.148464 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-6ddd31da-551b-46df-9003-c2eb21d275a2 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:41:02.150644 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-6ddd31da-551b-46df-9003-c2eb21d275a2 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:41:02.151462 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-6ddd31da-551b-46df-9003-c2eb21d275a2 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:41:02.322448 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-fb2b24e6-77cb-44ab-826b-fc3423a9e09d tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8118f0c698aa428a93cb054abac30c2e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f249d9041c6749f58973a5b0867b07ce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:41:02.336179 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-fb2b24e6-77cb-44ab-826b-fc3423a9e09d tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] 10.222.0.26 "GET /compute/v2.1/servers/e8a28aad-697f-4dee-8254-6f08e9d76168" status: 200 len: 1314 microversion: 2.1 time: 0.934508 Oct 19 00:41:02.350526 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 23/45] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:41:01 2022] GET /compute/v2.1/servers/e8a28aad-697f-4dee-8254-6f08e9d76168 => generated 1314 bytes in 940 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:41:02.351106 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-69f96bb9-e497-4238-9bf8-5bad72e2cb09 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:41:02.352447 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-69f96bb9-e497-4238-9bf8-5bad72e2cb09 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] [instance: ab4c73d6-9203-4110-93af-4a78428c672b] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:41:02.371682 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-69f96bb9-e497-4238-9bf8-5bad72e2cb09 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:41:02.372662 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-69f96bb9-e497-4238-9bf8-5bad72e2cb09 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:41:02.373844 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-69f96bb9-e497-4238-9bf8-5bad72e2cb09 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:41:02.377373 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-69f96bb9-e497-4238-9bf8-5bad72e2cb09 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:41:02.379819 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-69f96bb9-e497-4238-9bf8-5bad72e2cb09 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:41:02.382099 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-69f96bb9-e497-4238-9bf8-5bad72e2cb09 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:41:02.554444 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.policy [None req-6ddd31da-551b-46df-9003-c2eb21d275a2 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f8725c4d05b4dd79b05b43fba15980b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0b35c2038f6b4d0a9012b004f1416c00', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125640) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:41:02.566911 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: INFO nova.api.openstack.requestlog [None req-6ddd31da-551b-46df-9003-c2eb21d275a2 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] 10.222.0.26 "GET /compute/v2.1/servers/dc8b8a6e-eab7-4f45-8f5d-916c52eec398" status: 200 len: 1298 microversion: 2.1 time: 0.458107 Oct 19 00:41:02.584424 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: [pid: 125640|app: 0|req: 23/46] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:41:02 2022] GET /compute/v2.1/servers/dc8b8a6e-eab7-4f45-8f5d-916c52eec398 => generated 1298 bytes in 477 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:41:02.997251 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi [req-c1497d97-6e4a-4fd6-8c9b-00809cfea84b req-50c8bdc5-ab3a-42cb-a49a-2e8629a59e3b service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "3da61834-fd67-49d2-925c-c18fc264042c", "tag": "da1ae702-fcf6-4388-8f27-6c4ba5eb5fcf"}]} {{(pid=125640) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 19 00:41:03.007196 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-69f96bb9-e497-4238-9bf8-5bad72e2cb09 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd3f0d615187d4779bdd7da8c9f24c796', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0151d58abd047a692f5cc4e7d05bea5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:41:03.020996 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [req-c1497d97-6e4a-4fd6-8c9b-00809cfea84b req-50c8bdc5-ab3a-42cb-a49a-2e8629a59e3b service nova] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:41:03.025710 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [req-c1497d97-6e4a-4fd6-8c9b-00809cfea84b req-50c8bdc5-ab3a-42cb-a49a-2e8629a59e3b service nova] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:41:03.026165 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [req-c1497d97-6e4a-4fd6-8c9b-00809cfea84b req-50c8bdc5-ab3a-42cb-a49a-2e8629a59e3b service nova] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:41:03.029914 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-69f96bb9-e497-4238-9bf8-5bad72e2cb09 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] 10.222.0.26 "GET /compute/v2.1/servers/ab4c73d6-9203-4110-93af-4a78428c672b" status: 200 len: 1322 microversion: 2.1 time: 0.684783 Oct 19 00:41:03.032652 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 24/47] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:41:02 2022] GET /compute/v2.1/servers/ab4c73d6-9203-4110-93af-4a78428c672b => generated 1322 bytes in 690 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:41:03.049670 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG dbcounter [-] [125640] Writing DB stats nova_cell0:SELECT=26 {{(pid=125640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 00:41:03.081094 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: INFO nova.api.openstack.compute.server_external_events [req-c1497d97-6e4a-4fd6-8c9b-00809cfea84b req-50c8bdc5-ab3a-42cb-a49a-2e8629a59e3b service nova] Creating event network-changed:da1ae702-fcf6-4388-8f27-6c4ba5eb5fcf for instance 3da61834-fd67-49d2-925c-c18fc264042c on hv-cinder-maste Oct 19 00:41:03.082100 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [req-c1497d97-6e4a-4fd6-8c9b-00809cfea84b req-50c8bdc5-ab3a-42cb-a49a-2e8629a59e3b service nova] Acquiring lock "compute-rpcapi-router" {{(pid=125640) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:312}} Oct 19 00:41:03.082705 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [req-c1497d97-6e4a-4fd6-8c9b-00809cfea84b req-50c8bdc5-ab3a-42cb-a49a-2e8629a59e3b service nova] Acquired lock "compute-rpcapi-router" {{(pid=125640) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:315}} Oct 19 00:41:03.084176 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-63e7efdd-0fa6-4223-8c42-16074cd32056 None None] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:41:03.084854 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-63e7efdd-0fa6-4223-8c42-16074cd32056 None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:41:03.085641 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-63e7efdd-0fa6-4223-8c42-16074cd32056 None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:41:03.088582 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-63e7efdd-0fa6-4223-8c42-16074cd32056 None None] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:41:03.091143 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-63e7efdd-0fa6-4223-8c42-16074cd32056 None None] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:41:03.091578 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-63e7efdd-0fa6-4223-8c42-16074cd32056 None None] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:41:03.120352 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: INFO nova.compute.rpcapi [None req-63e7efdd-0fa6-4223-8c42-16074cd32056 None None] Automatically selected compute RPC version 6.1 from minimum service version 64 Oct 19 00:41:03.121274 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-63e7efdd-0fa6-4223-8c42-16074cd32056 None None] Releasing lock "compute-rpcapi-router" {{(pid=125640) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:333}} Oct 19 00:41:03.142352 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-3c74e3c4-db32-4aea-be51-f763ac664bbe tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:41:03.148296 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: INFO nova.api.openstack.requestlog [None req-63e7efdd-0fa6-4223-8c42-16074cd32056 None None] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.562145 Oct 19 00:41:03.148590 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-3c74e3c4-db32-4aea-be51-f763ac664bbe tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] [instance: b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:41:03.149321 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: [pid: 125640|app: 0|req: 24/48] 10.222.0.26 () {64 vars in 1317 bytes} [Wed Oct 19 00:41:02 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 564 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 19 00:41:03.181612 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-3c74e3c4-db32-4aea-be51-f763ac664bbe tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:41:03.184594 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-3c74e3c4-db32-4aea-be51-f763ac664bbe tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:41:03.188159 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-3c74e3c4-db32-4aea-be51-f763ac664bbe tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:41:03.190808 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-3c74e3c4-db32-4aea-be51-f763ac664bbe tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:41:03.191292 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-3c74e3c4-db32-4aea-be51-f763ac664bbe tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:41:03.191736 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-3c74e3c4-db32-4aea-be51-f763ac664bbe tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:41:03.264372 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG dbcounter [-] [125641] Writing DB stats nova_cell1:SELECT=100 {{(pid=125641) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 00:41:03.371049 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi [None req-0a01fb0f-1b60-4f4b-9fb5-cd5f9302c754 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Calling method '>' {{(pid=125640) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:41:03.372094 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.compute.api [None req-0a01fb0f-1b60-4f4b-9fb5-cd5f9302c754 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] [instance: e8a28aad-697f-4dee-8254-6f08e9d76168] Fetching instance by UUID {{(pid=125640) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:41:03.385558 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-0a01fb0f-1b60-4f4b-9fb5-cd5f9302c754 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:41:03.386812 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-0a01fb0f-1b60-4f4b-9fb5-cd5f9302c754 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:41:03.387483 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-0a01fb0f-1b60-4f4b-9fb5-cd5f9302c754 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:41:03.388444 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-0a01fb0f-1b60-4f4b-9fb5-cd5f9302c754 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:41:03.395810 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-0a01fb0f-1b60-4f4b-9fb5-cd5f9302c754 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:41:03.398239 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-0a01fb0f-1b60-4f4b-9fb5-cd5f9302c754 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:41:04.082174 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-3c74e3c4-db32-4aea-be51-f763ac664bbe tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:41:04.097396 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-3c74e3c4-db32-4aea-be51-f763ac664bbe tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:41:04.102760 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-3c74e3c4-db32-4aea-be51-f763ac664bbe tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:41:04.108077 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-3c74e3c4-db32-4aea-be51-f763ac664bbe tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] 10.222.0.26 "GET /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2" status: 200 len: 1407 microversion: 2.20 time: 0.981164 Oct 19 00:41:04.109351 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 25/49] 10.222.0.26 () {60 vars in 1277 bytes} [Wed Oct 19 00:41:03 2022] GET /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2 => generated 1407 bytes in 983 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 19 00:41:04.120893 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-129254f4-4452-4e3c-9332-41a728336a9b tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:41:04.128608 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-129254f4-4452-4e3c-9332-41a728336a9b tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] [instance: dc8b8a6e-eab7-4f45-8f5d-916c52eec398] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:41:04.152162 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-129254f4-4452-4e3c-9332-41a728336a9b tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:41:04.154035 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-129254f4-4452-4e3c-9332-41a728336a9b tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:41:04.155371 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-129254f4-4452-4e3c-9332-41a728336a9b tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:41:04.161322 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-129254f4-4452-4e3c-9332-41a728336a9b tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:41:04.162528 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-129254f4-4452-4e3c-9332-41a728336a9b tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:41:04.163271 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-129254f4-4452-4e3c-9332-41a728336a9b tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:41:04.391364 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.policy [None req-0a01fb0f-1b60-4f4b-9fb5-cd5f9302c754 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8118f0c698aa428a93cb054abac30c2e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f249d9041c6749f58973a5b0867b07ce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125640) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:41:04.403296 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: INFO nova.api.openstack.requestlog [None req-0a01fb0f-1b60-4f4b-9fb5-cd5f9302c754 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] 10.222.0.26 "GET /compute/v2.1/servers/e8a28aad-697f-4dee-8254-6f08e9d76168" status: 200 len: 1314 microversion: 2.1 time: 1.036365 Oct 19 00:41:04.404540 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: [pid: 125640|app: 0|req: 25/50] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:41:03 2022] GET /compute/v2.1/servers/e8a28aad-697f-4dee-8254-6f08e9d76168 => generated 1314 bytes in 1039 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 19 00:41:04.618144 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-129254f4-4452-4e3c-9332-41a728336a9b tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f8725c4d05b4dd79b05b43fba15980b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0b35c2038f6b4d0a9012b004f1416c00', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:41:04.626436 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-129254f4-4452-4e3c-9332-41a728336a9b tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] 10.222.0.26 "GET /compute/v2.1/servers/dc8b8a6e-eab7-4f45-8f5d-916c52eec398" status: 200 len: 1365 microversion: 2.1 time: 0.515824 Oct 19 00:41:04.649294 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 26/51] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:41:04 2022] GET /compute/v2.1/servers/dc8b8a6e-eab7-4f45-8f5d-916c52eec398 => generated 1365 bytes in 539 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:41:04.655342 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-567db04b-8aee-4971-9ea4-95743754419c tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:41:04.656191 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-567db04b-8aee-4971-9ea4-95743754419c tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] [instance: ab4c73d6-9203-4110-93af-4a78428c672b] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:41:04.663805 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-567db04b-8aee-4971-9ea4-95743754419c tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:41:04.664788 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-567db04b-8aee-4971-9ea4-95743754419c tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:41:04.665452 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-567db04b-8aee-4971-9ea4-95743754419c tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:41:04.666832 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-567db04b-8aee-4971-9ea4-95743754419c tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:41:04.668082 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-567db04b-8aee-4971-9ea4-95743754419c tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:41:04.668822 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-567db04b-8aee-4971-9ea4-95743754419c tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:41:04.859003 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi [req-aa035938-3544-4fd7-a6b6-1659a3f822d0 req-58bebc47-1eb8-41f3-9293-9c784d5c9d5c service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "ab4c73d6-9203-4110-93af-4a78428c672b", "tag": "cc5abfca-70ab-406b-974c-c89318ad195e"}]} {{(pid=125640) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 19 00:41:04.890815 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG dbcounter [-] [125640] Writing DB stats nova_api:SELECT=92,nova_api:INSERT=8 {{(pid=125640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 00:41:04.895700 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [req-aa035938-3544-4fd7-a6b6-1659a3f822d0 req-58bebc47-1eb8-41f3-9293-9c784d5c9d5c service nova] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:41:04.897591 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [req-aa035938-3544-4fd7-a6b6-1659a3f822d0 req-58bebc47-1eb8-41f3-9293-9c784d5c9d5c service nova] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:41:04.899209 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [req-aa035938-3544-4fd7-a6b6-1659a3f822d0 req-58bebc47-1eb8-41f3-9293-9c784d5c9d5c service nova] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:41:04.903164 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-567db04b-8aee-4971-9ea4-95743754419c tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd3f0d615187d4779bdd7da8c9f24c796', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0151d58abd047a692f5cc4e7d05bea5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:41:04.913010 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-567db04b-8aee-4971-9ea4-95743754419c tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] 10.222.0.26 "GET /compute/v2.1/servers/ab4c73d6-9203-4110-93af-4a78428c672b" status: 200 len: 1322 microversion: 2.1 time: 0.262054 Oct 19 00:41:04.914515 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 27/52] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:41:04 2022] GET /compute/v2.1/servers/ab4c73d6-9203-4110-93af-4a78428c672b => generated 1322 bytes in 264 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:41:04.953733 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: INFO nova.api.openstack.compute.server_external_events [req-aa035938-3544-4fd7-a6b6-1659a3f822d0 req-58bebc47-1eb8-41f3-9293-9c784d5c9d5c service nova] Creating event network-changed:cc5abfca-70ab-406b-974c-c89318ad195e for instance ab4c73d6-9203-4110-93af-4a78428c672b on hv-cinder-maste Oct 19 00:41:04.964339 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: INFO nova.api.openstack.requestlog [req-aa035938-3544-4fd7-a6b6-1659a3f822d0 req-58bebc47-1eb8-41f3-9293-9c784d5c9d5c service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.558306 Oct 19 00:41:04.966922 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: [pid: 125640|app: 0|req: 26/53] 10.222.0.26 () {64 vars in 1317 bytes} [Wed Oct 19 00:41:04 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 561 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 19 00:41:05.138330 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-799f9f25-e036-4982-a02c-08d340b423e0 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:41:05.141458 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-799f9f25-e036-4982-a02c-08d340b423e0 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] [instance: b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:41:05.172941 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-799f9f25-e036-4982-a02c-08d340b423e0 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:41:05.173798 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-799f9f25-e036-4982-a02c-08d340b423e0 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:41:05.175727 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-799f9f25-e036-4982-a02c-08d340b423e0 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:41:05.181792 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-799f9f25-e036-4982-a02c-08d340b423e0 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:41:05.183225 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-799f9f25-e036-4982-a02c-08d340b423e0 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:41:05.184542 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-799f9f25-e036-4982-a02c-08d340b423e0 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:41:05.214405 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi [req-7537a930-dbae-41bc-b08d-af49284cb4da req-91d0376b-f04f-455b-870e-2db7273e5bc8 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2", "tag": "6aea38b4-b7a0-4e8e-82dc-deb19c260092"}]} {{(pid=125640) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 19 00:41:05.230587 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [req-7537a930-dbae-41bc-b08d-af49284cb4da req-91d0376b-f04f-455b-870e-2db7273e5bc8 service nova] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:41:05.231313 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [req-7537a930-dbae-41bc-b08d-af49284cb4da req-91d0376b-f04f-455b-870e-2db7273e5bc8 service nova] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:41:05.232243 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [req-7537a930-dbae-41bc-b08d-af49284cb4da req-91d0376b-f04f-455b-870e-2db7273e5bc8 service nova] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:41:05.251336 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: INFO nova.api.openstack.compute.server_external_events [req-7537a930-dbae-41bc-b08d-af49284cb4da req-91d0376b-f04f-455b-870e-2db7273e5bc8 service nova] Creating event network-changed:6aea38b4-b7a0-4e8e-82dc-deb19c260092 for instance b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2 on hv-cinder-maste Oct 19 00:41:05.258121 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: INFO nova.api.openstack.requestlog [req-7537a930-dbae-41bc-b08d-af49284cb4da req-91d0376b-f04f-455b-870e-2db7273e5bc8 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.047578 Oct 19 00:41:05.259494 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: [pid: 125640|app: 0|req: 27/54] 10.222.0.26 () {64 vars in 1317 bytes} [Wed Oct 19 00:41:05 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 19 00:41:05.432126 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi [None req-2745aea2-0e6d-4685-8a78-4b4006562571 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Calling method '>' {{(pid=125640) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:41:05.434876 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.compute.api [None req-2745aea2-0e6d-4685-8a78-4b4006562571 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] [instance: e8a28aad-697f-4dee-8254-6f08e9d76168] Fetching instance by UUID {{(pid=125640) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:41:05.453426 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-2745aea2-0e6d-4685-8a78-4b4006562571 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:41:05.456082 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-2745aea2-0e6d-4685-8a78-4b4006562571 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:41:05.457953 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-2745aea2-0e6d-4685-8a78-4b4006562571 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:41:05.459040 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-2745aea2-0e6d-4685-8a78-4b4006562571 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:41:05.461851 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-2745aea2-0e6d-4685-8a78-4b4006562571 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:41:05.462465 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-2745aea2-0e6d-4685-8a78-4b4006562571 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:41:05.594319 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-799f9f25-e036-4982-a02c-08d340b423e0 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:41:05.605183 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-799f9f25-e036-4982-a02c-08d340b423e0 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:41:05.606805 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-799f9f25-e036-4982-a02c-08d340b423e0 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:41:05.608033 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-799f9f25-e036-4982-a02c-08d340b423e0 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] 10.222.0.26 "GET /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2" status: 200 len: 1407 microversion: 2.20 time: 0.476094 Oct 19 00:41:05.608704 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 28/55] 10.222.0.26 () {60 vars in 1277 bytes} [Wed Oct 19 00:41:05 2022] GET /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2 => generated 1407 bytes in 477 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 19 00:41:05.654998 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-b8579451-b2ed-4c45-866f-3f58b40246ad tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:41:05.655865 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-b8579451-b2ed-4c45-866f-3f58b40246ad tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] [instance: dc8b8a6e-eab7-4f45-8f5d-916c52eec398] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:41:05.666246 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-b8579451-b2ed-4c45-866f-3f58b40246ad tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:41:05.666832 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-b8579451-b2ed-4c45-866f-3f58b40246ad tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:41:05.667255 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-b8579451-b2ed-4c45-866f-3f58b40246ad tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:41:05.668000 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-b8579451-b2ed-4c45-866f-3f58b40246ad tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:41:05.668456 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-b8579451-b2ed-4c45-866f-3f58b40246ad tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:41:05.668881 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-b8579451-b2ed-4c45-866f-3f58b40246ad tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:41:05.848720 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-b8579451-b2ed-4c45-866f-3f58b40246ad tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f8725c4d05b4dd79b05b43fba15980b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0b35c2038f6b4d0a9012b004f1416c00', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:41:05.863893 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-b8579451-b2ed-4c45-866f-3f58b40246ad tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] 10.222.0.26 "GET /compute/v2.1/servers/dc8b8a6e-eab7-4f45-8f5d-916c52eec398" status: 200 len: 1365 microversion: 2.1 time: 0.212482 Oct 19 00:41:05.865360 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 29/56] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:41:05 2022] GET /compute/v2.1/servers/dc8b8a6e-eab7-4f45-8f5d-916c52eec398 => generated 1365 bytes in 215 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 19 00:41:05.939653 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-60db37e7-7aea-4808-ad70-dd41d649a49a tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:41:05.940455 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-60db37e7-7aea-4808-ad70-dd41d649a49a tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] [instance: ab4c73d6-9203-4110-93af-4a78428c672b] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:41:05.945707 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG dbcounter [-] [125641] Writing DB stats nova_api:SELECT=91,nova_api:INSERT=9 {{(pid=125641) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 00:41:05.947377 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-60db37e7-7aea-4808-ad70-dd41d649a49a tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:41:05.948178 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-60db37e7-7aea-4808-ad70-dd41d649a49a tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:41:05.948783 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-60db37e7-7aea-4808-ad70-dd41d649a49a tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:41:05.949657 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-60db37e7-7aea-4808-ad70-dd41d649a49a tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:41:05.950446 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-60db37e7-7aea-4808-ad70-dd41d649a49a tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:41:05.951059 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-60db37e7-7aea-4808-ad70-dd41d649a49a tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:41:06.060204 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.policy [None req-2745aea2-0e6d-4685-8a78-4b4006562571 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8118f0c698aa428a93cb054abac30c2e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f249d9041c6749f58973a5b0867b07ce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125640) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:41:06.070171 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: INFO nova.api.openstack.requestlog [None req-2745aea2-0e6d-4685-8a78-4b4006562571 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] 10.222.0.26 "GET /compute/v2.1/servers/e8a28aad-697f-4dee-8254-6f08e9d76168" status: 200 len: 1314 microversion: 2.1 time: 0.641790 Oct 19 00:41:06.086561 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: [pid: 125640|app: 0|req: 28/57] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:41:05 2022] GET /compute/v2.1/servers/e8a28aad-697f-4dee-8254-6f08e9d76168 => generated 1314 bytes in 660 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 19 00:41:06.229650 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-60db37e7-7aea-4808-ad70-dd41d649a49a tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd3f0d615187d4779bdd7da8c9f24c796', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0151d58abd047a692f5cc4e7d05bea5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:41:06.237905 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-60db37e7-7aea-4808-ad70-dd41d649a49a tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] 10.222.0.26 "GET /compute/v2.1/servers/ab4c73d6-9203-4110-93af-4a78428c672b" status: 200 len: 1322 microversion: 2.1 time: 0.302491 Oct 19 00:41:06.252157 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 30/58] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:41:05 2022] GET /compute/v2.1/servers/ab4c73d6-9203-4110-93af-4a78428c672b => generated 1322 bytes in 317 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 19 00:41:06.634101 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi [None req-28c9295a-4eee-4f21-9f1d-db8d28d27a04 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Calling method '>' {{(pid=125640) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:41:06.634798 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.compute.api [None req-28c9295a-4eee-4f21-9f1d-db8d28d27a04 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] [instance: b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2] Fetching instance by UUID {{(pid=125640) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:41:06.644560 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-28c9295a-4eee-4f21-9f1d-db8d28d27a04 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:41:06.645391 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-28c9295a-4eee-4f21-9f1d-db8d28d27a04 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:41:06.647489 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-28c9295a-4eee-4f21-9f1d-db8d28d27a04 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:41:06.649079 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-28c9295a-4eee-4f21-9f1d-db8d28d27a04 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:41:06.649813 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-28c9295a-4eee-4f21-9f1d-db8d28d27a04 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:41:06.651599 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-28c9295a-4eee-4f21-9f1d-db8d28d27a04 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:41:06.705982 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG dbcounter [-] [125640] Writing DB stats nova_cell1:SELECT=100 {{(pid=125640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 00:41:06.869713 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.policy [None req-28c9295a-4eee-4f21-9f1d-db8d28d27a04 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125640) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:41:06.880997 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.policy [None req-28c9295a-4eee-4f21-9f1d-db8d28d27a04 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125640) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:41:06.882927 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.policy [None req-28c9295a-4eee-4f21-9f1d-db8d28d27a04 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125640) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:41:06.884767 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: INFO nova.api.openstack.requestlog [None req-28c9295a-4eee-4f21-9f1d-db8d28d27a04 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] 10.222.0.26 "GET /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2" status: 200 len: 1407 microversion: 2.20 time: 0.254697 Oct 19 00:41:06.894558 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-612d35ce-ae51-4670-9363-943f95f5d9c9 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:41:06.894558 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-612d35ce-ae51-4670-9363-943f95f5d9c9 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] [instance: dc8b8a6e-eab7-4f45-8f5d-916c52eec398] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:41:06.895745 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: [pid: 125640|app: 0|req: 29/59] 10.222.0.26 () {60 vars in 1277 bytes} [Wed Oct 19 00:41:06 2022] GET /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2 => generated 1407 bytes in 266 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 19 00:41:06.903681 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-612d35ce-ae51-4670-9363-943f95f5d9c9 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:41:06.904313 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-612d35ce-ae51-4670-9363-943f95f5d9c9 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:41:06.904773 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-612d35ce-ae51-4670-9363-943f95f5d9c9 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:41:06.906162 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-612d35ce-ae51-4670-9363-943f95f5d9c9 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:41:06.906907 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-612d35ce-ae51-4670-9363-943f95f5d9c9 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:41:06.908270 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-612d35ce-ae51-4670-9363-943f95f5d9c9 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:41:07.108941 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi [None req-ebfe61b7-0672-474b-88ee-a0da7554d4d8 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Calling method '>' {{(pid=125640) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:41:07.109559 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.compute.api [None req-ebfe61b7-0672-474b-88ee-a0da7554d4d8 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] [instance: e8a28aad-697f-4dee-8254-6f08e9d76168] Fetching instance by UUID {{(pid=125640) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:41:07.118013 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-ebfe61b7-0672-474b-88ee-a0da7554d4d8 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:41:07.118527 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-ebfe61b7-0672-474b-88ee-a0da7554d4d8 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:41:07.118973 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-ebfe61b7-0672-474b-88ee-a0da7554d4d8 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:41:07.119668 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-ebfe61b7-0672-474b-88ee-a0da7554d4d8 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:41:07.120119 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-ebfe61b7-0672-474b-88ee-a0da7554d4d8 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:41:07.120544 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-ebfe61b7-0672-474b-88ee-a0da7554d4d8 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:41:07.151954 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-612d35ce-ae51-4670-9363-943f95f5d9c9 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f8725c4d05b4dd79b05b43fba15980b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0b35c2038f6b4d0a9012b004f1416c00', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:41:07.166087 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-612d35ce-ae51-4670-9363-943f95f5d9c9 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] 10.222.0.26 "GET /compute/v2.1/servers/dc8b8a6e-eab7-4f45-8f5d-916c52eec398" status: 200 len: 1365 microversion: 2.1 time: 0.276202 Oct 19 00:41:07.179351 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 31/60] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:41:06 2022] GET /compute/v2.1/servers/dc8b8a6e-eab7-4f45-8f5d-916c52eec398 => generated 1365 bytes in 294 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 19 00:41:07.191151 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [req-edc802db-7068-479b-8b3f-c8810eeee6b3 req-baccb408-35ac-4d16-96f2-b8fb47d4652f service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "e8a28aad-697f-4dee-8254-6f08e9d76168", "tag": "522599ca-04f8-4a03-91d1-860898570a3b"}]} {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 19 00:41:07.208056 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-edc802db-7068-479b-8b3f-c8810eeee6b3 req-baccb408-35ac-4d16-96f2-b8fb47d4652f service nova] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:41:07.208813 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-edc802db-7068-479b-8b3f-c8810eeee6b3 req-baccb408-35ac-4d16-96f2-b8fb47d4652f service nova] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:41:07.209443 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-edc802db-7068-479b-8b3f-c8810eeee6b3 req-baccb408-35ac-4d16-96f2-b8fb47d4652f service nova] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:41:07.233305 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.compute.server_external_events [req-edc802db-7068-479b-8b3f-c8810eeee6b3 req-baccb408-35ac-4d16-96f2-b8fb47d4652f service nova] Creating event network-changed:522599ca-04f8-4a03-91d1-860898570a3b for instance e8a28aad-697f-4dee-8254-6f08e9d76168 on hv-cinder-maste Oct 19 00:41:07.234434 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-edc802db-7068-479b-8b3f-c8810eeee6b3 req-baccb408-35ac-4d16-96f2-b8fb47d4652f service nova] Acquiring lock "compute-rpcapi-router" {{(pid=125641) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:312}} Oct 19 00:41:07.234991 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-edc802db-7068-479b-8b3f-c8810eeee6b3 req-baccb408-35ac-4d16-96f2-b8fb47d4652f service nova] Acquired lock "compute-rpcapi-router" {{(pid=125641) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:315}} Oct 19 00:41:07.236977 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-4a812069-e077-4669-af83-9b7961cf75c1 None None] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:41:07.237699 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-4a812069-e077-4669-af83-9b7961cf75c1 None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:41:07.238595 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-4a812069-e077-4669-af83-9b7961cf75c1 None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:41:07.239471 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-4a812069-e077-4669-af83-9b7961cf75c1 None None] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:41:07.240065 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-4a812069-e077-4669-af83-9b7961cf75c1 None None] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:41:07.241037 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-4a812069-e077-4669-af83-9b7961cf75c1 None None] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:41:07.254291 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.compute.rpcapi [None req-4a812069-e077-4669-af83-9b7961cf75c1 None None] Automatically selected compute RPC version 6.1 from minimum service version 64 Oct 19 00:41:07.255240 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-4a812069-e077-4669-af83-9b7961cf75c1 None None] Releasing lock "compute-rpcapi-router" {{(pid=125641) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:333}} Oct 19 00:41:07.274844 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-4a812069-e077-4669-af83-9b7961cf75c1 None None] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.094135 Oct 19 00:41:07.275924 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 32/61] 10.222.0.26 () {64 vars in 1317 bytes} [Wed Oct 19 00:41:07 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 19 00:41:07.285164 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-a966ec15-adcb-4ea4-925c-2d52dae3299c tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:41:07.286004 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-a966ec15-adcb-4ea4-925c-2d52dae3299c tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] [instance: ab4c73d6-9203-4110-93af-4a78428c672b] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:41:07.296088 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-a966ec15-adcb-4ea4-925c-2d52dae3299c tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:41:07.296577 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-a966ec15-adcb-4ea4-925c-2d52dae3299c tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:41:07.297013 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-a966ec15-adcb-4ea4-925c-2d52dae3299c tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:41:07.297709 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-a966ec15-adcb-4ea4-925c-2d52dae3299c tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:41:07.298163 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-a966ec15-adcb-4ea4-925c-2d52dae3299c tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:41:07.299685 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-a966ec15-adcb-4ea4-925c-2d52dae3299c tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:41:07.308788 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.policy [None req-ebfe61b7-0672-474b-88ee-a0da7554d4d8 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8118f0c698aa428a93cb054abac30c2e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f249d9041c6749f58973a5b0867b07ce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125640) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:41:07.317575 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: INFO nova.api.openstack.requestlog [None req-ebfe61b7-0672-474b-88ee-a0da7554d4d8 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] 10.222.0.26 "GET /compute/v2.1/servers/e8a28aad-697f-4dee-8254-6f08e9d76168" status: 200 len: 1314 microversion: 2.1 time: 0.212466 Oct 19 00:41:07.318355 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: [pid: 125640|app: 0|req: 30/62] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:41:07 2022] GET /compute/v2.1/servers/e8a28aad-697f-4dee-8254-6f08e9d76168 => generated 1314 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 19 00:41:07.347150 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi [req-6f3a8810-87bb-498a-9d06-d6d8b4c9ac97 req-dbf53160-203b-4197-9e80-14ed9158dec2 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "dc8b8a6e-eab7-4f45-8f5d-916c52eec398", "tag": "e00fb603-2d03-421e-b1f4-e3767c67aadc"}]} {{(pid=125640) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 19 00:41:07.359595 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [req-6f3a8810-87bb-498a-9d06-d6d8b4c9ac97 req-dbf53160-203b-4197-9e80-14ed9158dec2 service nova] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:41:07.360388 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [req-6f3a8810-87bb-498a-9d06-d6d8b4c9ac97 req-dbf53160-203b-4197-9e80-14ed9158dec2 service nova] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:41:07.360884 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [req-6f3a8810-87bb-498a-9d06-d6d8b4c9ac97 req-dbf53160-203b-4197-9e80-14ed9158dec2 service nova] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:41:07.375310 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: INFO nova.api.openstack.compute.server_external_events [req-6f3a8810-87bb-498a-9d06-d6d8b4c9ac97 req-dbf53160-203b-4197-9e80-14ed9158dec2 service nova] Creating event network-changed:e00fb603-2d03-421e-b1f4-e3767c67aadc for instance dc8b8a6e-eab7-4f45-8f5d-916c52eec398 on hv-cinder-maste Oct 19 00:41:07.383791 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: INFO nova.api.openstack.requestlog [req-6f3a8810-87bb-498a-9d06-d6d8b4c9ac97 req-dbf53160-203b-4197-9e80-14ed9158dec2 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.041352 Oct 19 00:41:07.384702 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: [pid: 125640|app: 0|req: 31/63] 10.222.0.26 () {64 vars in 1317 bytes} [Wed Oct 19 00:41:07 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 19 00:41:07.466809 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-a966ec15-adcb-4ea4-925c-2d52dae3299c tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd3f0d615187d4779bdd7da8c9f24c796', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0151d58abd047a692f5cc4e7d05bea5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:41:07.477014 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-a966ec15-adcb-4ea4-925c-2d52dae3299c tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] 10.222.0.26 "GET /compute/v2.1/servers/ab4c73d6-9203-4110-93af-4a78428c672b" status: 200 len: 1322 microversion: 2.1 time: 0.199683 Oct 19 00:41:07.491566 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 33/64] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:41:07 2022] GET /compute/v2.1/servers/ab4c73d6-9203-4110-93af-4a78428c672b => generated 1322 bytes in 215 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:41:07.914588 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi [None req-006d4c2c-0755-46fc-bd2e-aea2b924093b tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Calling method '>' {{(pid=125640) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:41:07.916005 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.compute.api [None req-006d4c2c-0755-46fc-bd2e-aea2b924093b tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] [instance: b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2] Fetching instance by UUID {{(pid=125640) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:41:07.927754 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-006d4c2c-0755-46fc-bd2e-aea2b924093b tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:41:07.928537 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-006d4c2c-0755-46fc-bd2e-aea2b924093b tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:41:07.929756 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-006d4c2c-0755-46fc-bd2e-aea2b924093b tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:41:07.931213 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-006d4c2c-0755-46fc-bd2e-aea2b924093b tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:41:07.932502 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-006d4c2c-0755-46fc-bd2e-aea2b924093b tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:41:07.933160 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-006d4c2c-0755-46fc-bd2e-aea2b924093b tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:41:08.193906 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-287831d0-c7e0-4d34-9993-3f26f6b4dbba tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:41:08.194435 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-287831d0-c7e0-4d34-9993-3f26f6b4dbba tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] [instance: dc8b8a6e-eab7-4f45-8f5d-916c52eec398] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:41:08.207680 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-287831d0-c7e0-4d34-9993-3f26f6b4dbba tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:41:08.208349 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-287831d0-c7e0-4d34-9993-3f26f6b4dbba tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:41:08.208657 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-287831d0-c7e0-4d34-9993-3f26f6b4dbba tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:41:08.213951 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-287831d0-c7e0-4d34-9993-3f26f6b4dbba tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:41:08.214527 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-287831d0-c7e0-4d34-9993-3f26f6b4dbba tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:41:08.214985 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-287831d0-c7e0-4d34-9993-3f26f6b4dbba tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:41:08.556550 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-287831d0-c7e0-4d34-9993-3f26f6b4dbba tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f8725c4d05b4dd79b05b43fba15980b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0b35c2038f6b4d0a9012b004f1416c00', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:41:08.573756 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-287831d0-c7e0-4d34-9993-3f26f6b4dbba tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] 10.222.0.26 "GET /compute/v2.1/servers/dc8b8a6e-eab7-4f45-8f5d-916c52eec398" status: 200 len: 1365 microversion: 2.1 time: 0.384102 Oct 19 00:41:08.575523 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 34/65] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:41:08 2022] GET /compute/v2.1/servers/dc8b8a6e-eab7-4f45-8f5d-916c52eec398 => generated 1365 bytes in 387 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 19 00:41:08.584321 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-6319243a-7fe5-443e-b359-5ab48fd7de20 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:41:08.585423 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-6319243a-7fe5-443e-b359-5ab48fd7de20 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] [instance: e8a28aad-697f-4dee-8254-6f08e9d76168] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:41:08.594060 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-6319243a-7fe5-443e-b359-5ab48fd7de20 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:41:08.595152 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-6319243a-7fe5-443e-b359-5ab48fd7de20 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:41:08.596097 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-6319243a-7fe5-443e-b359-5ab48fd7de20 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:41:08.597439 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-6319243a-7fe5-443e-b359-5ab48fd7de20 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:41:08.603424 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-6319243a-7fe5-443e-b359-5ab48fd7de20 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:41:08.603886 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-6319243a-7fe5-443e-b359-5ab48fd7de20 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:41:08.618896 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.policy [None req-006d4c2c-0755-46fc-bd2e-aea2b924093b tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125640) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:41:08.627378 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.policy [None req-006d4c2c-0755-46fc-bd2e-aea2b924093b tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125640) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:41:08.628866 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.policy [None req-006d4c2c-0755-46fc-bd2e-aea2b924093b tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125640) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:41:08.634673 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: INFO nova.api.openstack.requestlog [None req-006d4c2c-0755-46fc-bd2e-aea2b924093b tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] 10.222.0.26 "GET /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2" status: 200 len: 1407 microversion: 2.20 time: 0.723030 Oct 19 00:41:08.636736 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: [pid: 125640|app: 0|req: 32/66] 10.222.0.26 () {60 vars in 1277 bytes} [Wed Oct 19 00:41:07 2022] GET /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2 => generated 1407 bytes in 728 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 19 00:41:08.644100 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi [None req-1ca8f1f3-0cb0-4041-b1f2-c6c99f4cecd2 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Calling method '>' {{(pid=125640) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:41:08.644987 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.compute.api [None req-1ca8f1f3-0cb0-4041-b1f2-c6c99f4cecd2 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] [instance: ab4c73d6-9203-4110-93af-4a78428c672b] Fetching instance by UUID {{(pid=125640) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:41:08.674217 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-1ca8f1f3-0cb0-4041-b1f2-c6c99f4cecd2 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:41:08.676250 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-1ca8f1f3-0cb0-4041-b1f2-c6c99f4cecd2 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:41:08.676850 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-1ca8f1f3-0cb0-4041-b1f2-c6c99f4cecd2 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:41:08.677930 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-1ca8f1f3-0cb0-4041-b1f2-c6c99f4cecd2 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:41:08.678583 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-1ca8f1f3-0cb0-4041-b1f2-c6c99f4cecd2 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:41:08.679171 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-1ca8f1f3-0cb0-4041-b1f2-c6c99f4cecd2 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:41:08.832200 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-6319243a-7fe5-443e-b359-5ab48fd7de20 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8118f0c698aa428a93cb054abac30c2e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f249d9041c6749f58973a5b0867b07ce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:41:08.840901 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-6319243a-7fe5-443e-b359-5ab48fd7de20 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] 10.222.0.26 "GET /compute/v2.1/servers/e8a28aad-697f-4dee-8254-6f08e9d76168" status: 200 len: 1314 microversion: 2.1 time: 0.263649 Oct 19 00:41:08.841480 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 35/67] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:41:08 2022] GET /compute/v2.1/servers/e8a28aad-697f-4dee-8254-6f08e9d76168 => generated 1314 bytes in 265 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:41:08.953502 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.policy [None req-1ca8f1f3-0cb0-4041-b1f2-c6c99f4cecd2 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd3f0d615187d4779bdd7da8c9f24c796', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0151d58abd047a692f5cc4e7d05bea5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125640) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:41:08.965360 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: INFO nova.api.openstack.requestlog [None req-1ca8f1f3-0cb0-4041-b1f2-c6c99f4cecd2 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] 10.222.0.26 "GET /compute/v2.1/servers/ab4c73d6-9203-4110-93af-4a78428c672b" status: 200 len: 1322 microversion: 2.1 time: 0.326298 Oct 19 00:41:08.965951 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: [pid: 125640|app: 0|req: 33/68] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:41:08 2022] GET /compute/v2.1/servers/ab4c73d6-9203-4110-93af-4a78428c672b => generated 1322 bytes in 328 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:41:09.597656 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-6a507e06-6585-444e-94a7-d899ff8d8648 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:41:09.598282 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-6a507e06-6585-444e-94a7-d899ff8d8648 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] [instance: dc8b8a6e-eab7-4f45-8f5d-916c52eec398] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:41:09.606585 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-6a507e06-6585-444e-94a7-d899ff8d8648 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:41:09.610528 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-6a507e06-6585-444e-94a7-d899ff8d8648 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:41:09.611254 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-6a507e06-6585-444e-94a7-d899ff8d8648 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:41:09.614116 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-6a507e06-6585-444e-94a7-d899ff8d8648 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:41:09.615097 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-6a507e06-6585-444e-94a7-d899ff8d8648 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:41:09.615802 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-6a507e06-6585-444e-94a7-d899ff8d8648 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:41:09.666161 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi [None req-f92c85a1-7050-424a-a943-0b70524b8fc7 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Calling method '>' {{(pid=125640) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:41:09.666883 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.compute.api [None req-f92c85a1-7050-424a-a943-0b70524b8fc7 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] [instance: b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2] Fetching instance by UUID {{(pid=125640) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:41:09.683507 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-f92c85a1-7050-424a-a943-0b70524b8fc7 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:41:09.684205 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-f92c85a1-7050-424a-a943-0b70524b8fc7 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:41:09.684838 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-f92c85a1-7050-424a-a943-0b70524b8fc7 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:41:09.686006 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-f92c85a1-7050-424a-a943-0b70524b8fc7 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:41:09.687938 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-f92c85a1-7050-424a-a943-0b70524b8fc7 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:41:09.688720 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-f92c85a1-7050-424a-a943-0b70524b8fc7 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:41:10.077845 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-6a507e06-6585-444e-94a7-d899ff8d8648 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f8725c4d05b4dd79b05b43fba15980b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0b35c2038f6b4d0a9012b004f1416c00', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:41:10.090806 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-6a507e06-6585-444e-94a7-d899ff8d8648 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] 10.222.0.26 "GET /compute/v2.1/servers/dc8b8a6e-eab7-4f45-8f5d-916c52eec398" status: 200 len: 1365 microversion: 2.1 time: 0.497224 Oct 19 00:41:10.091409 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 36/69] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:41:09 2022] GET /compute/v2.1/servers/dc8b8a6e-eab7-4f45-8f5d-916c52eec398 => generated 1365 bytes in 499 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:41:10.104696 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-de2605d7-f27f-4969-a72a-86f770797b81 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:41:10.105526 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-de2605d7-f27f-4969-a72a-86f770797b81 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] [instance: e8a28aad-697f-4dee-8254-6f08e9d76168] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:41:10.122920 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-de2605d7-f27f-4969-a72a-86f770797b81 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:41:10.123410 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-de2605d7-f27f-4969-a72a-86f770797b81 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:41:10.123847 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-de2605d7-f27f-4969-a72a-86f770797b81 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:41:10.124591 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-de2605d7-f27f-4969-a72a-86f770797b81 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:41:10.125055 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-de2605d7-f27f-4969-a72a-86f770797b81 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:41:10.125475 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-de2605d7-f27f-4969-a72a-86f770797b81 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:41:10.219381 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.policy [None req-f92c85a1-7050-424a-a943-0b70524b8fc7 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125640) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:41:10.230086 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.policy [None req-f92c85a1-7050-424a-a943-0b70524b8fc7 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125640) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:41:10.236456 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.policy [None req-f92c85a1-7050-424a-a943-0b70524b8fc7 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125640) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:41:10.241712 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: INFO nova.api.openstack.requestlog [None req-f92c85a1-7050-424a-a943-0b70524b8fc7 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] 10.222.0.26 "GET /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2" status: 200 len: 1407 microversion: 2.20 time: 0.579603 Oct 19 00:41:10.242010 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: [pid: 125640|app: 0|req: 34/70] 10.222.0.26 () {60 vars in 1277 bytes} [Wed Oct 19 00:41:09 2022] GET /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2 => generated 1407 bytes in 581 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 19 00:41:10.251496 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi [None req-a5f035cb-e61f-4b33-b840-f06e737558fb tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Calling method '>' {{(pid=125640) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:41:10.252236 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.compute.api [None req-a5f035cb-e61f-4b33-b840-f06e737558fb tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] [instance: ab4c73d6-9203-4110-93af-4a78428c672b] Fetching instance by UUID {{(pid=125640) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:41:10.267987 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-a5f035cb-e61f-4b33-b840-f06e737558fb tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:41:10.268600 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-a5f035cb-e61f-4b33-b840-f06e737558fb tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:41:10.268881 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-a5f035cb-e61f-4b33-b840-f06e737558fb tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:41:10.269594 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-a5f035cb-e61f-4b33-b840-f06e737558fb tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:41:10.270055 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-a5f035cb-e61f-4b33-b840-f06e737558fb tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:41:10.270503 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-a5f035cb-e61f-4b33-b840-f06e737558fb tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:41:10.385309 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-de2605d7-f27f-4969-a72a-86f770797b81 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8118f0c698aa428a93cb054abac30c2e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f249d9041c6749f58973a5b0867b07ce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:41:10.398398 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-de2605d7-f27f-4969-a72a-86f770797b81 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] 10.222.0.26 "GET /compute/v2.1/servers/e8a28aad-697f-4dee-8254-6f08e9d76168" status: 200 len: 1314 microversion: 2.1 time: 0.298609 Oct 19 00:41:10.429792 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 37/71] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:41:10 2022] GET /compute/v2.1/servers/e8a28aad-697f-4dee-8254-6f08e9d76168 => generated 1314 bytes in 331 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:41:10.544767 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.policy [None req-a5f035cb-e61f-4b33-b840-f06e737558fb tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd3f0d615187d4779bdd7da8c9f24c796', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0151d58abd047a692f5cc4e7d05bea5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125640) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:41:10.598600 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: INFO nova.api.openstack.requestlog [None req-a5f035cb-e61f-4b33-b840-f06e737558fb tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] 10.222.0.26 "GET /compute/v2.1/servers/ab4c73d6-9203-4110-93af-4a78428c672b" status: 200 len: 1322 microversion: 2.1 time: 0.355125 Oct 19 00:41:10.622451 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: [pid: 125640|app: 0|req: 35/72] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:41:10 2022] GET /compute/v2.1/servers/ab4c73d6-9203-4110-93af-4a78428c672b => generated 1322 bytes in 369 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:41:11.135187 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi [None req-8a6f9625-e336-4b5f-ae47-f9f36c189a38 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Calling method '>' {{(pid=125640) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:41:11.137044 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.compute.api [None req-8a6f9625-e336-4b5f-ae47-f9f36c189a38 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] [instance: dc8b8a6e-eab7-4f45-8f5d-916c52eec398] Fetching instance by UUID {{(pid=125640) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:41:11.150289 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-8a6f9625-e336-4b5f-ae47-f9f36c189a38 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:41:11.151981 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-8a6f9625-e336-4b5f-ae47-f9f36c189a38 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:41:11.152610 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-8a6f9625-e336-4b5f-ae47-f9f36c189a38 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:41:11.153567 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-8a6f9625-e336-4b5f-ae47-f9f36c189a38 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:41:11.154213 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-8a6f9625-e336-4b5f-ae47-f9f36c189a38 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:41:11.154990 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-8a6f9625-e336-4b5f-ae47-f9f36c189a38 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:41:11.567292 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.policy [None req-8a6f9625-e336-4b5f-ae47-f9f36c189a38 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f8725c4d05b4dd79b05b43fba15980b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0b35c2038f6b4d0a9012b004f1416c00', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125640) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:41:11.583195 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: INFO nova.api.openstack.requestlog [None req-8a6f9625-e336-4b5f-ae47-f9f36c189a38 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] 10.222.0.26 "GET /compute/v2.1/servers/dc8b8a6e-eab7-4f45-8f5d-916c52eec398" status: 200 len: 1365 microversion: 2.1 time: 0.462920 Oct 19 00:41:11.586963 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [req-3047192b-9b29-4cbf-a423-0313f26177b7 req-1a8d809e-97c0-465e-84b8-d86991c88fed service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "3da61834-fd67-49d2-925c-c18fc264042c", "name": "network-vif-plugged", "status": "completed", "tag": "da1ae702-fcf6-4388-8f27-6c4ba5eb5fcf"}]} {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 19 00:41:11.587807 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: [pid: 125640|app: 0|req: 36/73] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:41:11 2022] GET /compute/v2.1/servers/dc8b8a6e-eab7-4f45-8f5d-916c52eec398 => generated 1365 bytes in 470 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:41:11.595904 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi [None req-3d794a6d-122f-4d4b-9228-87be448008e1 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Calling method '>' {{(pid=125640) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:41:11.605048 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-3047192b-9b29-4cbf-a423-0313f26177b7 req-1a8d809e-97c0-465e-84b8-d86991c88fed service nova] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:41:11.605537 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-3047192b-9b29-4cbf-a423-0313f26177b7 req-1a8d809e-97c0-465e-84b8-d86991c88fed service nova] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:41:11.605970 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-3047192b-9b29-4cbf-a423-0313f26177b7 req-1a8d809e-97c0-465e-84b8-d86991c88fed service nova] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:41:11.613644 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.compute.api [None req-3d794a6d-122f-4d4b-9228-87be448008e1 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] [instance: b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2] Fetching instance by UUID {{(pid=125640) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:41:11.633674 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-3d794a6d-122f-4d4b-9228-87be448008e1 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:41:11.634720 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.compute.server_external_events [req-3047192b-9b29-4cbf-a423-0313f26177b7 req-1a8d809e-97c0-465e-84b8-d86991c88fed service nova] Creating event network-vif-plugged:da1ae702-fcf6-4388-8f27-6c4ba5eb5fcf for instance 3da61834-fd67-49d2-925c-c18fc264042c on hv-cinder-maste Oct 19 00:41:11.635341 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-3d794a6d-122f-4d4b-9228-87be448008e1 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:41:11.636720 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-3d794a6d-122f-4d4b-9228-87be448008e1 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:41:11.639649 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-3d794a6d-122f-4d4b-9228-87be448008e1 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:41:11.640333 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-3d794a6d-122f-4d4b-9228-87be448008e1 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:41:11.640952 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-3d794a6d-122f-4d4b-9228-87be448008e1 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:41:11.647093 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [req-3047192b-9b29-4cbf-a423-0313f26177b7 req-1a8d809e-97c0-465e-84b8-d86991c88fed service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.585607 Oct 19 00:41:11.648704 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 38/74] 10.222.0.26 () {64 vars in 1317 bytes} [Wed Oct 19 00:41:11 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 589 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 19 00:41:11.657352 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-562faa90-3bd3-457e-8559-bf863cf876ee tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:41:11.658809 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-562faa90-3bd3-457e-8559-bf863cf876ee tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] [instance: e8a28aad-697f-4dee-8254-6f08e9d76168] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:41:11.667286 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-562faa90-3bd3-457e-8559-bf863cf876ee tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:41:11.668016 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-562faa90-3bd3-457e-8559-bf863cf876ee tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:41:11.668822 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-562faa90-3bd3-457e-8559-bf863cf876ee tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:41:11.669865 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-562faa90-3bd3-457e-8559-bf863cf876ee tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:41:11.671206 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-562faa90-3bd3-457e-8559-bf863cf876ee tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:41:11.671873 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-562faa90-3bd3-457e-8559-bf863cf876ee tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:41:11.841444 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.policy [None req-3d794a6d-122f-4d4b-9228-87be448008e1 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125640) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:41:11.854152 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.policy [None req-3d794a6d-122f-4d4b-9228-87be448008e1 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125640) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:41:11.855866 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.policy [None req-3d794a6d-122f-4d4b-9228-87be448008e1 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125640) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:41:11.858022 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: INFO nova.api.openstack.requestlog [None req-3d794a6d-122f-4d4b-9228-87be448008e1 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] 10.222.0.26 "GET /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2" status: 200 len: 1407 microversion: 2.20 time: 0.267792 Oct 19 00:41:11.859107 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: [pid: 125640|app: 0|req: 37/75] 10.222.0.26 () {60 vars in 1277 bytes} [Wed Oct 19 00:41:11 2022] GET /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2 => generated 1407 bytes in 271 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 19 00:41:11.885889 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi [None req-e4da5c60-8797-42ef-9383-427d04a78c5f tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Calling method '>' {{(pid=125640) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:41:11.886691 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.compute.api [None req-e4da5c60-8797-42ef-9383-427d04a78c5f tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] [instance: ab4c73d6-9203-4110-93af-4a78428c672b] Fetching instance by UUID {{(pid=125640) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:41:11.895467 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-e4da5c60-8797-42ef-9383-427d04a78c5f tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:41:11.896028 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-e4da5c60-8797-42ef-9383-427d04a78c5f tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:41:11.896468 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-e4da5c60-8797-42ef-9383-427d04a78c5f tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:41:11.897166 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-e4da5c60-8797-42ef-9383-427d04a78c5f tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:41:11.897778 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-e4da5c60-8797-42ef-9383-427d04a78c5f tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:41:11.898275 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-e4da5c60-8797-42ef-9383-427d04a78c5f tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:41:11.919674 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-562faa90-3bd3-457e-8559-bf863cf876ee tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8118f0c698aa428a93cb054abac30c2e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f249d9041c6749f58973a5b0867b07ce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:41:11.928937 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-562faa90-3bd3-457e-8559-bf863cf876ee tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] 10.222.0.26 "GET /compute/v2.1/servers/e8a28aad-697f-4dee-8254-6f08e9d76168" status: 200 len: 1314 microversion: 2.1 time: 0.278678 Oct 19 00:41:11.943257 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 39/76] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:41:11 2022] GET /compute/v2.1/servers/e8a28aad-697f-4dee-8254-6f08e9d76168 => generated 1314 bytes in 294 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:41:12.196099 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.policy [None req-e4da5c60-8797-42ef-9383-427d04a78c5f tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd3f0d615187d4779bdd7da8c9f24c796', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0151d58abd047a692f5cc4e7d05bea5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125640) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:41:12.211669 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: INFO nova.api.openstack.requestlog [None req-e4da5c60-8797-42ef-9383-427d04a78c5f tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] 10.222.0.26 "GET /compute/v2.1/servers/ab4c73d6-9203-4110-93af-4a78428c672b" status: 200 len: 1322 microversion: 2.1 time: 0.337528 Oct 19 00:41:12.212576 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: [pid: 125640|app: 0|req: 38/77] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:41:11 2022] GET /compute/v2.1/servers/ab4c73d6-9203-4110-93af-4a78428c672b => generated 1322 bytes in 339 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:41:12.609576 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-02c26a56-0d87-44ae-a056-36f346f7864b tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:41:12.611703 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-02c26a56-0d87-44ae-a056-36f346f7864b tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] [instance: dc8b8a6e-eab7-4f45-8f5d-916c52eec398] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:41:12.621765 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-02c26a56-0d87-44ae-a056-36f346f7864b tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:41:12.629900 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-02c26a56-0d87-44ae-a056-36f346f7864b tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:41:12.630978 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-02c26a56-0d87-44ae-a056-36f346f7864b tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:41:12.635204 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-02c26a56-0d87-44ae-a056-36f346f7864b tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:41:12.636314 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-02c26a56-0d87-44ae-a056-36f346f7864b tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:41:12.641278 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-02c26a56-0d87-44ae-a056-36f346f7864b tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:41:12.892241 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi [None req-004330eb-79a9-4a65-881f-de52c22e9ed8 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Calling method '>' {{(pid=125640) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:41:12.893652 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.compute.api [None req-004330eb-79a9-4a65-881f-de52c22e9ed8 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] [instance: b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2] Fetching instance by UUID {{(pid=125640) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:41:12.901247 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-004330eb-79a9-4a65-881f-de52c22e9ed8 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:41:12.902160 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-004330eb-79a9-4a65-881f-de52c22e9ed8 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:41:12.903968 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-004330eb-79a9-4a65-881f-de52c22e9ed8 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:41:12.905002 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-004330eb-79a9-4a65-881f-de52c22e9ed8 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:41:12.906181 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-004330eb-79a9-4a65-881f-de52c22e9ed8 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:41:12.907034 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-004330eb-79a9-4a65-881f-de52c22e9ed8 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:41:12.936463 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-02c26a56-0d87-44ae-a056-36f346f7864b tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f8725c4d05b4dd79b05b43fba15980b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0b35c2038f6b4d0a9012b004f1416c00', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:41:12.949996 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-02c26a56-0d87-44ae-a056-36f346f7864b tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] 10.222.0.26 "GET /compute/v2.1/servers/dc8b8a6e-eab7-4f45-8f5d-916c52eec398" status: 200 len: 1365 microversion: 2.1 time: 0.341615 Oct 19 00:41:12.975277 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 40/78] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:41:12 2022] GET /compute/v2.1/servers/dc8b8a6e-eab7-4f45-8f5d-916c52eec398 => generated 1365 bytes in 369 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:41:12.988029 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-a85556a5-0f7a-4674-8733-5303d4e2d911 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:41:12.993294 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-a85556a5-0f7a-4674-8733-5303d4e2d911 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] [instance: e8a28aad-697f-4dee-8254-6f08e9d76168] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:41:13.010311 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-a85556a5-0f7a-4674-8733-5303d4e2d911 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:41:13.011541 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-a85556a5-0f7a-4674-8733-5303d4e2d911 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:41:13.014363 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-a85556a5-0f7a-4674-8733-5303d4e2d911 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:41:13.018113 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-a85556a5-0f7a-4674-8733-5303d4e2d911 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:41:13.018905 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-a85556a5-0f7a-4674-8733-5303d4e2d911 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:41:13.019623 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-a85556a5-0f7a-4674-8733-5303d4e2d911 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:41:13.107546 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG dbcounter [-] [125640] Writing DB stats nova_cell0:SELECT=2 {{(pid=125640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 00:41:13.314361 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-a85556a5-0f7a-4674-8733-5303d4e2d911 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8118f0c698aa428a93cb054abac30c2e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f249d9041c6749f58973a5b0867b07ce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:41:13.327856 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-a85556a5-0f7a-4674-8733-5303d4e2d911 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] 10.222.0.26 "GET /compute/v2.1/servers/e8a28aad-697f-4dee-8254-6f08e9d76168" status: 200 len: 1314 microversion: 2.1 time: 0.350791 Oct 19 00:41:13.328858 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 41/79] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:41:12 2022] GET /compute/v2.1/servers/e8a28aad-697f-4dee-8254-6f08e9d76168 => generated 1314 bytes in 353 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:41:13.334825 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-1a2eff98-253a-41c2-884b-f97136edf466 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:41:13.335648 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-1a2eff98-253a-41c2-884b-f97136edf466 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] [instance: ab4c73d6-9203-4110-93af-4a78428c672b] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:41:13.342249 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-1a2eff98-253a-41c2-884b-f97136edf466 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:41:13.342922 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-1a2eff98-253a-41c2-884b-f97136edf466 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:41:13.343789 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-1a2eff98-253a-41c2-884b-f97136edf466 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:41:13.344854 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-1a2eff98-253a-41c2-884b-f97136edf466 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:41:13.346218 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-1a2eff98-253a-41c2-884b-f97136edf466 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:41:13.346896 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-1a2eff98-253a-41c2-884b-f97136edf466 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:41:13.375572 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.policy [None req-004330eb-79a9-4a65-881f-de52c22e9ed8 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125640) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:41:13.394142 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.policy [None req-004330eb-79a9-4a65-881f-de52c22e9ed8 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125640) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:41:13.398880 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.policy [None req-004330eb-79a9-4a65-881f-de52c22e9ed8 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125640) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:41:13.406830 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: INFO nova.api.openstack.requestlog [None req-004330eb-79a9-4a65-881f-de52c22e9ed8 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] 10.222.0.26 "GET /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2" status: 200 len: 1407 microversion: 2.20 time: 0.513998 Oct 19 00:41:13.412153 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: [pid: 125640|app: 0|req: 39/80] 10.222.0.26 () {60 vars in 1277 bytes} [Wed Oct 19 00:41:12 2022] GET /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2 => generated 1407 bytes in 527 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 19 00:41:13.535595 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-1a2eff98-253a-41c2-884b-f97136edf466 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd3f0d615187d4779bdd7da8c9f24c796', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0151d58abd047a692f5cc4e7d05bea5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:41:13.544757 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-1a2eff98-253a-41c2-884b-f97136edf466 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] 10.222.0.26 "GET /compute/v2.1/servers/ab4c73d6-9203-4110-93af-4a78428c672b" status: 200 len: 1322 microversion: 2.1 time: 0.214603 Oct 19 00:41:13.545934 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 42/81] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:41:13 2022] GET /compute/v2.1/servers/ab4c73d6-9203-4110-93af-4a78428c672b => generated 1322 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:41:13.671703 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi [req-d0fb57c5-677c-44e2-80d7-ec33485d92c4 req-c08476d2-2559-4c46-b395-d9098be2ba89 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "ab4c73d6-9203-4110-93af-4a78428c672b", "name": "network-vif-plugged", "status": "completed", "tag": "cc5abfca-70ab-406b-974c-c89318ad195e"}]} {{(pid=125640) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 19 00:41:13.684629 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [req-d0fb57c5-677c-44e2-80d7-ec33485d92c4 req-c08476d2-2559-4c46-b395-d9098be2ba89 service nova] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:41:13.685650 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [req-d0fb57c5-677c-44e2-80d7-ec33485d92c4 req-c08476d2-2559-4c46-b395-d9098be2ba89 service nova] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:41:13.686307 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [req-d0fb57c5-677c-44e2-80d7-ec33485d92c4 req-c08476d2-2559-4c46-b395-d9098be2ba89 service nova] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:41:13.702585 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: INFO nova.api.openstack.compute.server_external_events [req-d0fb57c5-677c-44e2-80d7-ec33485d92c4 req-c08476d2-2559-4c46-b395-d9098be2ba89 service nova] Creating event network-vif-plugged:cc5abfca-70ab-406b-974c-c89318ad195e for instance ab4c73d6-9203-4110-93af-4a78428c672b on hv-cinder-maste Oct 19 00:41:13.711505 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: INFO nova.api.openstack.requestlog [req-d0fb57c5-677c-44e2-80d7-ec33485d92c4 req-c08476d2-2559-4c46-b395-d9098be2ba89 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.046420 Oct 19 00:41:13.712396 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: [pid: 125640|app: 0|req: 40/82] 10.222.0.26 () {64 vars in 1317 bytes} [Wed Oct 19 00:41:13 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 19 00:41:13.974129 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-b2c03207-4268-41e0-969e-339e4ac427a9 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:41:13.974886 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-b2c03207-4268-41e0-969e-339e4ac427a9 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] [instance: dc8b8a6e-eab7-4f45-8f5d-916c52eec398] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:41:13.983448 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-b2c03207-4268-41e0-969e-339e4ac427a9 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:41:13.983448 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-b2c03207-4268-41e0-969e-339e4ac427a9 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:41:13.983448 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-b2c03207-4268-41e0-969e-339e4ac427a9 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:41:13.985083 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-b2c03207-4268-41e0-969e-339e4ac427a9 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:41:13.986071 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-b2c03207-4268-41e0-969e-339e4ac427a9 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:41:13.986660 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-b2c03207-4268-41e0-969e-339e4ac427a9 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:41:13.999994 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG dbcounter [-] [125641] Writing DB stats nova_cell1:SELECT=100 {{(pid=125641) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 00:41:14.195325 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-b2c03207-4268-41e0-969e-339e4ac427a9 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f8725c4d05b4dd79b05b43fba15980b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0b35c2038f6b4d0a9012b004f1416c00', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:41:14.206061 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-b2c03207-4268-41e0-969e-339e4ac427a9 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] 10.222.0.26 "GET /compute/v2.1/servers/dc8b8a6e-eab7-4f45-8f5d-916c52eec398" status: 200 len: 1365 microversion: 2.1 time: 0.235717 Oct 19 00:41:14.208048 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 43/83] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:41:13 2022] GET /compute/v2.1/servers/dc8b8a6e-eab7-4f45-8f5d-916c52eec398 => generated 1365 bytes in 239 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:41:14.362771 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi [None req-6f15e651-1eac-4175-8785-57577ec34d6b tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Calling method '>' {{(pid=125640) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:41:14.363906 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.compute.api [None req-6f15e651-1eac-4175-8785-57577ec34d6b tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] [instance: e8a28aad-697f-4dee-8254-6f08e9d76168] Fetching instance by UUID {{(pid=125640) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:41:14.374525 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-6f15e651-1eac-4175-8785-57577ec34d6b tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:41:14.376947 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-6f15e651-1eac-4175-8785-57577ec34d6b tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:41:14.378016 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-6f15e651-1eac-4175-8785-57577ec34d6b tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:41:14.379874 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-6f15e651-1eac-4175-8785-57577ec34d6b tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:41:14.381028 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-6f15e651-1eac-4175-8785-57577ec34d6b tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:41:14.383137 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-6f15e651-1eac-4175-8785-57577ec34d6b tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:41:14.441575 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-b3a77de1-f82e-46a9-b30c-93957e527b47 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:41:14.442726 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-b3a77de1-f82e-46a9-b30c-93957e527b47 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] [instance: b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:41:14.451255 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-b3a77de1-f82e-46a9-b30c-93957e527b47 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:41:14.452297 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-b3a77de1-f82e-46a9-b30c-93957e527b47 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:41:14.452953 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-b3a77de1-f82e-46a9-b30c-93957e527b47 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:41:14.454071 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-b3a77de1-f82e-46a9-b30c-93957e527b47 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:41:14.455778 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-b3a77de1-f82e-46a9-b30c-93957e527b47 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:41:14.456621 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-b3a77de1-f82e-46a9-b30c-93957e527b47 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:41:14.613412 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.policy [None req-6f15e651-1eac-4175-8785-57577ec34d6b tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8118f0c698aa428a93cb054abac30c2e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f249d9041c6749f58973a5b0867b07ce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125640) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:41:14.629313 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: INFO nova.api.openstack.requestlog [None req-6f15e651-1eac-4175-8785-57577ec34d6b tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] 10.222.0.26 "GET /compute/v2.1/servers/e8a28aad-697f-4dee-8254-6f08e9d76168" status: 200 len: 1314 microversion: 2.1 time: 0.269354 Oct 19 00:41:14.630156 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: [pid: 125640|app: 0|req: 41/84] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:41:14 2022] GET /compute/v2.1/servers/e8a28aad-697f-4dee-8254-6f08e9d76168 => generated 1314 bytes in 276 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:41:14.637113 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi [None req-7f81c52e-98b4-450f-8d87-0ca5239d3ef0 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Calling method '>' {{(pid=125640) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:41:14.638034 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.compute.api [None req-7f81c52e-98b4-450f-8d87-0ca5239d3ef0 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] [instance: ab4c73d6-9203-4110-93af-4a78428c672b] Fetching instance by UUID {{(pid=125640) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:41:14.652345 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-7f81c52e-98b4-450f-8d87-0ca5239d3ef0 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:41:14.653255 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-7f81c52e-98b4-450f-8d87-0ca5239d3ef0 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:41:14.654305 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-7f81c52e-98b4-450f-8d87-0ca5239d3ef0 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:41:14.655642 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-7f81c52e-98b4-450f-8d87-0ca5239d3ef0 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:41:14.656661 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-7f81c52e-98b4-450f-8d87-0ca5239d3ef0 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:41:14.657292 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-7f81c52e-98b4-450f-8d87-0ca5239d3ef0 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:41:14.735366 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-b3a77de1-f82e-46a9-b30c-93957e527b47 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:41:14.750712 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-b3a77de1-f82e-46a9-b30c-93957e527b47 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:41:14.766188 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-b3a77de1-f82e-46a9-b30c-93957e527b47 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:41:14.767931 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-b3a77de1-f82e-46a9-b30c-93957e527b47 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] 10.222.0.26 "GET /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2" status: 200 len: 1407 microversion: 2.20 time: 0.332357 Oct 19 00:41:14.769143 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 44/85] 10.222.0.26 () {60 vars in 1277 bytes} [Wed Oct 19 00:41:14 2022] GET /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2 => generated 1407 bytes in 334 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 19 00:41:14.846362 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.policy [None req-7f81c52e-98b4-450f-8d87-0ca5239d3ef0 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd3f0d615187d4779bdd7da8c9f24c796', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0151d58abd047a692f5cc4e7d05bea5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125640) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:41:14.855416 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: INFO nova.api.openstack.requestlog [None req-7f81c52e-98b4-450f-8d87-0ca5239d3ef0 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] 10.222.0.26 "GET /compute/v2.1/servers/ab4c73d6-9203-4110-93af-4a78428c672b" status: 200 len: 1322 microversion: 2.1 time: 0.223898 Oct 19 00:41:14.856161 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: [pid: 125640|app: 0|req: 42/86] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:41:14 2022] GET /compute/v2.1/servers/ab4c73d6-9203-4110-93af-4a78428c672b => generated 1322 bytes in 225 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:41:15.233016 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-a3a6db99-4ae2-4e09-957f-1b92beedcfba tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:41:15.234827 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-a3a6db99-4ae2-4e09-957f-1b92beedcfba tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] [instance: dc8b8a6e-eab7-4f45-8f5d-916c52eec398] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:41:15.244222 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-a3a6db99-4ae2-4e09-957f-1b92beedcfba tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:41:15.244698 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-a3a6db99-4ae2-4e09-957f-1b92beedcfba tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:41:15.245140 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-a3a6db99-4ae2-4e09-957f-1b92beedcfba tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:41:15.245835 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-a3a6db99-4ae2-4e09-957f-1b92beedcfba tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:41:15.246284 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-a3a6db99-4ae2-4e09-957f-1b92beedcfba tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:41:15.246731 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-a3a6db99-4ae2-4e09-957f-1b92beedcfba tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:41:15.535167 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-a3a6db99-4ae2-4e09-957f-1b92beedcfba tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f8725c4d05b4dd79b05b43fba15980b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0b35c2038f6b4d0a9012b004f1416c00', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:41:15.551053 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-a3a6db99-4ae2-4e09-957f-1b92beedcfba tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] 10.222.0.26 "GET /compute/v2.1/servers/dc8b8a6e-eab7-4f45-8f5d-916c52eec398" status: 200 len: 1365 microversion: 2.1 time: 0.318881 Oct 19 00:41:15.551053 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 45/87] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:41:15 2022] GET /compute/v2.1/servers/dc8b8a6e-eab7-4f45-8f5d-916c52eec398 => generated 1365 bytes in 320 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:41:15.656991 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi [None req-44fc3ad7-7ad1-47d6-b26c-f9b3cc12724a tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Calling method '>' {{(pid=125640) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:41:15.658245 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.compute.api [None req-44fc3ad7-7ad1-47d6-b26c-f9b3cc12724a tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] [instance: e8a28aad-697f-4dee-8254-6f08e9d76168] Fetching instance by UUID {{(pid=125640) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:41:15.670119 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-44fc3ad7-7ad1-47d6-b26c-f9b3cc12724a tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:41:15.671180 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-44fc3ad7-7ad1-47d6-b26c-f9b3cc12724a tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:41:15.671910 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-44fc3ad7-7ad1-47d6-b26c-f9b3cc12724a tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:41:15.675860 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-44fc3ad7-7ad1-47d6-b26c-f9b3cc12724a tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:41:15.676867 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-44fc3ad7-7ad1-47d6-b26c-f9b3cc12724a tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:41:15.677457 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-44fc3ad7-7ad1-47d6-b26c-f9b3cc12724a tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:41:15.741798 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [req-965b1348-b494-4389-9b51-8fd1604e05c4 req-16f98cf4-0dcc-4a0f-a6ee-16be5a79de98 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2", "name": "network-vif-plugged", "status": "completed", "tag": "6aea38b4-b7a0-4e8e-82dc-deb19c260092"}, {"server_uuid": "e8a28aad-697f-4dee-8254-6f08e9d76168", "name": "network-vif-plugged", "status": "completed", "tag": "522599ca-04f8-4a03-91d1-860898570a3b"}]} {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 19 00:41:15.778940 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-965b1348-b494-4389-9b51-8fd1604e05c4 req-16f98cf4-0dcc-4a0f-a6ee-16be5a79de98 service nova] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:41:15.781125 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-965b1348-b494-4389-9b51-8fd1604e05c4 req-16f98cf4-0dcc-4a0f-a6ee-16be5a79de98 service nova] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:41:15.785163 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-965b1348-b494-4389-9b51-8fd1604e05c4 req-16f98cf4-0dcc-4a0f-a6ee-16be5a79de98 service nova] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:41:15.819646 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.compute.server_external_events [req-965b1348-b494-4389-9b51-8fd1604e05c4 req-16f98cf4-0dcc-4a0f-a6ee-16be5a79de98 service nova] Creating event network-vif-plugged:6aea38b4-b7a0-4e8e-82dc-deb19c260092 for instance b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2 on hv-cinder-maste Oct 19 00:41:15.820346 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.compute.server_external_events [req-965b1348-b494-4389-9b51-8fd1604e05c4 req-16f98cf4-0dcc-4a0f-a6ee-16be5a79de98 service nova] Creating event network-vif-plugged:522599ca-04f8-4a03-91d1-860898570a3b for instance e8a28aad-697f-4dee-8254-6f08e9d76168 on hv-cinder-maste Oct 19 00:41:15.839007 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [req-965b1348-b494-4389-9b51-8fd1604e05c4 req-16f98cf4-0dcc-4a0f-a6ee-16be5a79de98 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 354 microversion: 2.1 time: 0.100700 Oct 19 00:41:15.840432 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 46/88] 10.222.0.26 () {64 vars in 1317 bytes} [Wed Oct 19 00:41:15 2022] POST /compute/v2.1/os-server-external-events => generated 354 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 19 00:41:15.848856 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-9d2ccbe7-79af-4196-8d34-2177d0795216 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:41:15.849797 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-9d2ccbe7-79af-4196-8d34-2177d0795216 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] [instance: b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:41:15.863819 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-9d2ccbe7-79af-4196-8d34-2177d0795216 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:41:15.863819 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-9d2ccbe7-79af-4196-8d34-2177d0795216 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:41:15.863819 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-9d2ccbe7-79af-4196-8d34-2177d0795216 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:41:15.863819 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-9d2ccbe7-79af-4196-8d34-2177d0795216 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:41:15.864152 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-9d2ccbe7-79af-4196-8d34-2177d0795216 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:41:15.864152 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-9d2ccbe7-79af-4196-8d34-2177d0795216 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:41:15.921697 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.policy [None req-44fc3ad7-7ad1-47d6-b26c-f9b3cc12724a tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8118f0c698aa428a93cb054abac30c2e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f249d9041c6749f58973a5b0867b07ce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125640) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:41:15.936724 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: INFO nova.api.openstack.requestlog [None req-44fc3ad7-7ad1-47d6-b26c-f9b3cc12724a tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] 10.222.0.26 "GET /compute/v2.1/servers/e8a28aad-697f-4dee-8254-6f08e9d76168" status: 200 len: 1314 microversion: 2.1 time: 0.282228 Oct 19 00:41:15.937549 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: [pid: 125640|app: 0|req: 43/89] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:41:15 2022] GET /compute/v2.1/servers/e8a28aad-697f-4dee-8254-6f08e9d76168 => generated 1314 bytes in 287 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:41:15.968374 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi [None req-5b4d5b80-daf3-49d6-be00-9e0a85c2bded tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Calling method '>' {{(pid=125640) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:41:15.969710 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.compute.api [None req-5b4d5b80-daf3-49d6-be00-9e0a85c2bded tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] [instance: ab4c73d6-9203-4110-93af-4a78428c672b] Fetching instance by UUID {{(pid=125640) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:41:15.983350 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-5b4d5b80-daf3-49d6-be00-9e0a85c2bded tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:41:15.984177 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-5b4d5b80-daf3-49d6-be00-9e0a85c2bded tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:41:15.984921 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-5b4d5b80-daf3-49d6-be00-9e0a85c2bded tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:41:15.986880 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-5b4d5b80-daf3-49d6-be00-9e0a85c2bded tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:41:15.988471 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-5b4d5b80-daf3-49d6-be00-9e0a85c2bded tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:41:15.989218 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-5b4d5b80-daf3-49d6-be00-9e0a85c2bded tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:41:16.150513 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-9d2ccbe7-79af-4196-8d34-2177d0795216 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:41:16.165462 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-9d2ccbe7-79af-4196-8d34-2177d0795216 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:41:16.189445 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-9d2ccbe7-79af-4196-8d34-2177d0795216 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:41:16.190795 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-9d2ccbe7-79af-4196-8d34-2177d0795216 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] 10.222.0.26 "GET /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2" status: 200 len: 1407 microversion: 2.20 time: 0.348411 Oct 19 00:41:16.197013 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 47/90] 10.222.0.26 () {60 vars in 1277 bytes} [Wed Oct 19 00:41:15 2022] GET /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2 => generated 1407 bytes in 355 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 19 00:41:16.209328 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.policy [None req-5b4d5b80-daf3-49d6-be00-9e0a85c2bded tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd3f0d615187d4779bdd7da8c9f24c796', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0151d58abd047a692f5cc4e7d05bea5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125640) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:41:16.221343 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: INFO nova.api.openstack.requestlog [None req-5b4d5b80-daf3-49d6-be00-9e0a85c2bded tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] 10.222.0.26 "GET /compute/v2.1/servers/ab4c73d6-9203-4110-93af-4a78428c672b" status: 200 len: 1322 microversion: 2.1 time: 0.268674 Oct 19 00:41:16.221929 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: [pid: 125640|app: 0|req: 44/91] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:41:15 2022] GET /compute/v2.1/servers/ab4c73d6-9203-4110-93af-4a78428c672b => generated 1322 bytes in 284 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:41:16.573965 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-1918f02a-9aa1-4535-bc4a-17bef8e7112d tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:41:16.574630 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-1918f02a-9aa1-4535-bc4a-17bef8e7112d tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] [instance: dc8b8a6e-eab7-4f45-8f5d-916c52eec398] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:41:16.581267 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-1918f02a-9aa1-4535-bc4a-17bef8e7112d tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:41:16.581746 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-1918f02a-9aa1-4535-bc4a-17bef8e7112d tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:41:16.582177 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-1918f02a-9aa1-4535-bc4a-17bef8e7112d tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:41:16.582901 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-1918f02a-9aa1-4535-bc4a-17bef8e7112d tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:41:16.583486 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-1918f02a-9aa1-4535-bc4a-17bef8e7112d tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:41:16.583913 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-1918f02a-9aa1-4535-bc4a-17bef8e7112d tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:41:16.717144 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-1918f02a-9aa1-4535-bc4a-17bef8e7112d tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f8725c4d05b4dd79b05b43fba15980b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0b35c2038f6b4d0a9012b004f1416c00', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:41:16.726003 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-1918f02a-9aa1-4535-bc4a-17bef8e7112d tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] 10.222.0.26 "GET /compute/v2.1/servers/dc8b8a6e-eab7-4f45-8f5d-916c52eec398" status: 200 len: 1365 microversion: 2.1 time: 0.156368 Oct 19 00:41:16.726590 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 48/92] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:41:16 2022] GET /compute/v2.1/servers/dc8b8a6e-eab7-4f45-8f5d-916c52eec398 => generated 1365 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:41:16.963582 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi [None req-ac57c2a9-0b01-4eef-b6bb-19bb88d32ccf tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Calling method '>' {{(pid=125640) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:41:16.964216 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.compute.api [None req-ac57c2a9-0b01-4eef-b6bb-19bb88d32ccf tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] [instance: e8a28aad-697f-4dee-8254-6f08e9d76168] Fetching instance by UUID {{(pid=125640) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:41:16.970299 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-ac57c2a9-0b01-4eef-b6bb-19bb88d32ccf tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:41:16.971302 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-ac57c2a9-0b01-4eef-b6bb-19bb88d32ccf tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:41:16.971529 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-ac57c2a9-0b01-4eef-b6bb-19bb88d32ccf tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:41:16.972235 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-ac57c2a9-0b01-4eef-b6bb-19bb88d32ccf tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:41:16.972693 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-ac57c2a9-0b01-4eef-b6bb-19bb88d32ccf tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:41:16.973121 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-ac57c2a9-0b01-4eef-b6bb-19bb88d32ccf tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:41:17.124352 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.policy [None req-ac57c2a9-0b01-4eef-b6bb-19bb88d32ccf tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8118f0c698aa428a93cb054abac30c2e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f249d9041c6749f58973a5b0867b07ce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125640) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:41:17.133131 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: INFO nova.api.openstack.requestlog [None req-ac57c2a9-0b01-4eef-b6bb-19bb88d32ccf tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] 10.222.0.26 "GET /compute/v2.1/servers/e8a28aad-697f-4dee-8254-6f08e9d76168" status: 200 len: 1314 microversion: 2.1 time: 0.173024 Oct 19 00:41:17.133716 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: [pid: 125640|app: 0|req: 45/93] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:41:16 2022] GET /compute/v2.1/servers/e8a28aad-697f-4dee-8254-6f08e9d76168 => generated 1314 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:41:17.219666 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-c12fe7c5-3a13-48f5-b996-4d49991bdf9c tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:41:17.221716 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-c12fe7c5-3a13-48f5-b996-4d49991bdf9c tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] [instance: b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:41:17.243205 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-c12fe7c5-3a13-48f5-b996-4d49991bdf9c tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:41:17.243778 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-c12fe7c5-3a13-48f5-b996-4d49991bdf9c tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:41:17.244283 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-c12fe7c5-3a13-48f5-b996-4d49991bdf9c tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:41:17.245183 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-c12fe7c5-3a13-48f5-b996-4d49991bdf9c tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:41:17.245840 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-c12fe7c5-3a13-48f5-b996-4d49991bdf9c tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:41:17.246464 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-c12fe7c5-3a13-48f5-b996-4d49991bdf9c tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:41:17.249971 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi [None req-1201c0af-ae41-4b95-998f-2e91a9489b68 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Calling method '>' {{(pid=125640) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:41:17.250624 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.compute.api [None req-1201c0af-ae41-4b95-998f-2e91a9489b68 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] [instance: ab4c73d6-9203-4110-93af-4a78428c672b] Fetching instance by UUID {{(pid=125640) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:41:17.253055 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG dbcounter [-] [125641] Writing DB stats nova_cell0:SELECT=2 {{(pid=125641) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 00:41:17.258070 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-1201c0af-ae41-4b95-998f-2e91a9489b68 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:41:17.258724 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-1201c0af-ae41-4b95-998f-2e91a9489b68 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:41:17.259362 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-1201c0af-ae41-4b95-998f-2e91a9489b68 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:41:17.260168 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-1201c0af-ae41-4b95-998f-2e91a9489b68 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:41:17.260803 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-1201c0af-ae41-4b95-998f-2e91a9489b68 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:41:17.261148 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-1201c0af-ae41-4b95-998f-2e91a9489b68 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:41:17.412528 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.policy [None req-1201c0af-ae41-4b95-998f-2e91a9489b68 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd3f0d615187d4779bdd7da8c9f24c796', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0151d58abd047a692f5cc4e7d05bea5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125640) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:41:17.421411 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: INFO nova.api.openstack.requestlog [None req-1201c0af-ae41-4b95-998f-2e91a9489b68 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] 10.222.0.26 "GET /compute/v2.1/servers/ab4c73d6-9203-4110-93af-4a78428c672b" status: 200 len: 1516 microversion: 2.1 time: 0.176880 Oct 19 00:41:17.434899 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: [pid: 125640|app: 0|req: 46/94] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:41:17 2022] GET /compute/v2.1/servers/ab4c73d6-9203-4110-93af-4a78428c672b => generated 1516 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 19 00:41:17.434991 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-c12fe7c5-3a13-48f5-b996-4d49991bdf9c tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:41:17.434991 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-c12fe7c5-3a13-48f5-b996-4d49991bdf9c tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:41:17.435910 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-c12fe7c5-3a13-48f5-b996-4d49991bdf9c tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:41:17.437572 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-c12fe7c5-3a13-48f5-b996-4d49991bdf9c tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] 10.222.0.26 "GET /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2" status: 200 len: 1407 microversion: 2.20 time: 0.225584 Oct 19 00:41:17.438320 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 49/95] 10.222.0.26 () {60 vars in 1277 bytes} [Wed Oct 19 00:41:17 2022] GET /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2 => generated 1407 bytes in 227 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 19 00:41:17.449023 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi [None req-681de65a-3e2c-4fa0-966f-87a7354303f7 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Action: 'action', calling method: >, body: {"rescue": {"adminPass": "***"}} {{(pid=125640) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 19 00:41:17.451384 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.compute.api [None req-681de65a-3e2c-4fa0-966f-87a7354303f7 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] [instance: ab4c73d6-9203-4110-93af-4a78428c672b] Fetching instance by UUID {{(pid=125640) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:41:17.461980 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-681de65a-3e2c-4fa0-966f-87a7354303f7 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:41:17.462466 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-681de65a-3e2c-4fa0-966f-87a7354303f7 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:41:17.462953 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-681de65a-3e2c-4fa0-966f-87a7354303f7 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:41:17.463700 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-681de65a-3e2c-4fa0-966f-87a7354303f7 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:41:17.464154 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-681de65a-3e2c-4fa0-966f-87a7354303f7 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:41:17.464575 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-681de65a-3e2c-4fa0-966f-87a7354303f7 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:41:17.521505 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG dbcounter [-] [125640] Writing DB stats nova_cell1:SELECT=100 {{(pid=125640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 00:41:17.529283 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.objects.instance [None req-681de65a-3e2c-4fa0-966f-87a7354303f7 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lazy-loading 'flavor' on Instance uuid ab4c73d6-9203-4110-93af-4a78428c672b {{(pid=125640) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 19 00:41:17.649058 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: INFO nova.api.openstack.requestlog [None req-681de65a-3e2c-4fa0-966f-87a7354303f7 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] 10.222.0.26 "POST /compute/v2.1/servers/ab4c73d6-9203-4110-93af-4a78428c672b/action" status: 200 len: 32 microversion: 2.1 time: 0.204342 Oct 19 00:41:17.649875 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: [pid: 125640|app: 0|req: 47/96] 10.222.0.26 () {60 vars in 1278 bytes} [Wed Oct 19 00:41:17 2022] POST /compute/v2.1/servers/ab4c73d6-9203-4110-93af-4a78428c672b/action => generated 32 bytes in 206 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Oct 19 00:41:17.659715 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-25e3c267-2b03-4050-a75c-c878243aab54 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:41:17.660423 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-25e3c267-2b03-4050-a75c-c878243aab54 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] [instance: ab4c73d6-9203-4110-93af-4a78428c672b] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:41:17.666572 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-25e3c267-2b03-4050-a75c-c878243aab54 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:41:17.667124 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-25e3c267-2b03-4050-a75c-c878243aab54 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:41:17.667624 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-25e3c267-2b03-4050-a75c-c878243aab54 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:41:17.668474 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-25e3c267-2b03-4050-a75c-c878243aab54 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:41:17.669018 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-25e3c267-2b03-4050-a75c-c878243aab54 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:41:17.669501 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-25e3c267-2b03-4050-a75c-c878243aab54 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:41:17.750259 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi [None req-9029d658-dd3b-419a-a80d-93de166f181d tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Calling method '>' {{(pid=125640) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:41:17.751100 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.compute.api [None req-9029d658-dd3b-419a-a80d-93de166f181d tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] [instance: dc8b8a6e-eab7-4f45-8f5d-916c52eec398] Fetching instance by UUID {{(pid=125640) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:41:17.758164 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-9029d658-dd3b-419a-a80d-93de166f181d tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:41:17.758824 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-9029d658-dd3b-419a-a80d-93de166f181d tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:41:17.759567 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-9029d658-dd3b-419a-a80d-93de166f181d tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:41:17.760664 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-9029d658-dd3b-419a-a80d-93de166f181d tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:41:17.761181 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-9029d658-dd3b-419a-a80d-93de166f181d tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:41:17.761716 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-9029d658-dd3b-419a-a80d-93de166f181d tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:41:17.822614 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-25e3c267-2b03-4050-a75c-c878243aab54 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd3f0d615187d4779bdd7da8c9f24c796', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0151d58abd047a692f5cc4e7d05bea5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:41:17.830838 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-25e3c267-2b03-4050-a75c-c878243aab54 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] 10.222.0.26 "GET /compute/v2.1/servers/ab4c73d6-9203-4110-93af-4a78428c672b" status: 200 len: 1522 microversion: 2.1 time: 0.174886 Oct 19 00:41:17.838998 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 50/97] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:41:17 2022] GET /compute/v2.1/servers/ab4c73d6-9203-4110-93af-4a78428c672b => generated 1522 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:41:17.906971 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.policy [None req-9029d658-dd3b-419a-a80d-93de166f181d tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f8725c4d05b4dd79b05b43fba15980b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0b35c2038f6b4d0a9012b004f1416c00', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125640) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:41:17.917085 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: INFO nova.api.openstack.requestlog [None req-9029d658-dd3b-419a-a80d-93de166f181d tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] 10.222.0.26 "GET /compute/v2.1/servers/dc8b8a6e-eab7-4f45-8f5d-916c52eec398" status: 200 len: 1365 microversion: 2.1 time: 0.171381 Oct 19 00:41:17.917703 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: [pid: 125640|app: 0|req: 48/98] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:41:17 2022] GET /compute/v2.1/servers/dc8b8a6e-eab7-4f45-8f5d-916c52eec398 => generated 1365 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 19 00:41:18.158053 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-07eef089-5403-4ad9-b581-a2ba074f6d01 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:41:18.158771 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-07eef089-5403-4ad9-b581-a2ba074f6d01 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] [instance: e8a28aad-697f-4dee-8254-6f08e9d76168] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:41:18.164860 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-07eef089-5403-4ad9-b581-a2ba074f6d01 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:41:18.165342 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-07eef089-5403-4ad9-b581-a2ba074f6d01 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:41:18.165773 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-07eef089-5403-4ad9-b581-a2ba074f6d01 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:41:18.166458 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-07eef089-5403-4ad9-b581-a2ba074f6d01 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:41:18.166923 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-07eef089-5403-4ad9-b581-a2ba074f6d01 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:41:18.167335 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-07eef089-5403-4ad9-b581-a2ba074f6d01 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:41:18.317654 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-07eef089-5403-4ad9-b581-a2ba074f6d01 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8118f0c698aa428a93cb054abac30c2e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f249d9041c6749f58973a5b0867b07ce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:41:18.326157 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-07eef089-5403-4ad9-b581-a2ba074f6d01 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] 10.222.0.26 "GET /compute/v2.1/servers/e8a28aad-697f-4dee-8254-6f08e9d76168" status: 200 len: 1314 microversion: 2.1 time: 0.171807 Oct 19 00:41:18.326759 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 51/99] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:41:18 2022] GET /compute/v2.1/servers/e8a28aad-697f-4dee-8254-6f08e9d76168 => generated 1314 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:41:18.465116 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi [None req-c6f394cd-d08a-408a-b32c-909b99e03a46 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Calling method '>' {{(pid=125640) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:41:18.465810 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.compute.api [None req-c6f394cd-d08a-408a-b32c-909b99e03a46 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] [instance: b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2] Fetching instance by UUID {{(pid=125640) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:41:18.474808 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-c6f394cd-d08a-408a-b32c-909b99e03a46 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:41:18.475455 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-c6f394cd-d08a-408a-b32c-909b99e03a46 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:41:18.476030 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-c6f394cd-d08a-408a-b32c-909b99e03a46 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:41:18.476803 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-c6f394cd-d08a-408a-b32c-909b99e03a46 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:41:18.477385 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-c6f394cd-d08a-408a-b32c-909b99e03a46 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:41:18.480939 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-c6f394cd-d08a-408a-b32c-909b99e03a46 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:41:18.625890 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.policy [None req-c6f394cd-d08a-408a-b32c-909b99e03a46 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125640) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:41:18.638649 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.policy [None req-c6f394cd-d08a-408a-b32c-909b99e03a46 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125640) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:41:18.640125 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.policy [None req-c6f394cd-d08a-408a-b32c-909b99e03a46 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125640) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:41:18.641227 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: INFO nova.api.openstack.requestlog [None req-c6f394cd-d08a-408a-b32c-909b99e03a46 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] 10.222.0.26 "GET /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2" status: 200 len: 1407 microversion: 2.20 time: 0.180076 Oct 19 00:41:18.642097 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: [pid: 125640|app: 0|req: 49/100] 10.222.0.26 () {60 vars in 1277 bytes} [Wed Oct 19 00:41:18 2022] GET /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2 => generated 1407 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 19 00:41:18.854421 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-d46cb135-bd02-4405-adc4-f476dfb3a013 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:41:18.855093 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-d46cb135-bd02-4405-adc4-f476dfb3a013 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] [instance: ab4c73d6-9203-4110-93af-4a78428c672b] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:41:18.861664 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d46cb135-bd02-4405-adc4-f476dfb3a013 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:41:18.862148 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d46cb135-bd02-4405-adc4-f476dfb3a013 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:41:18.862600 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d46cb135-bd02-4405-adc4-f476dfb3a013 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:41:18.863291 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d46cb135-bd02-4405-adc4-f476dfb3a013 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:41:18.863739 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d46cb135-bd02-4405-adc4-f476dfb3a013 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:41:18.864158 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d46cb135-bd02-4405-adc4-f476dfb3a013 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:41:18.941247 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi [None req-e5d180f2-0ffb-4d8b-b0fb-91e1cd51a682 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Calling method '>' {{(pid=125640) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:41:18.942159 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.compute.api [None req-e5d180f2-0ffb-4d8b-b0fb-91e1cd51a682 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] [instance: dc8b8a6e-eab7-4f45-8f5d-916c52eec398] Fetching instance by UUID {{(pid=125640) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:41:18.949047 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-e5d180f2-0ffb-4d8b-b0fb-91e1cd51a682 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:41:18.949540 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-e5d180f2-0ffb-4d8b-b0fb-91e1cd51a682 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:41:18.949977 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-e5d180f2-0ffb-4d8b-b0fb-91e1cd51a682 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:41:18.950723 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-e5d180f2-0ffb-4d8b-b0fb-91e1cd51a682 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:41:18.951187 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-e5d180f2-0ffb-4d8b-b0fb-91e1cd51a682 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:41:18.951593 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-e5d180f2-0ffb-4d8b-b0fb-91e1cd51a682 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:41:19.021504 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-d46cb135-bd02-4405-adc4-f476dfb3a013 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd3f0d615187d4779bdd7da8c9f24c796', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0151d58abd047a692f5cc4e7d05bea5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:41:19.034003 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-d46cb135-bd02-4405-adc4-f476dfb3a013 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] 10.222.0.26 "GET /compute/v2.1/servers/ab4c73d6-9203-4110-93af-4a78428c672b" status: 200 len: 1522 microversion: 2.1 time: 0.183136 Oct 19 00:41:19.034912 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 52/101] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:41:18 2022] GET /compute/v2.1/servers/ab4c73d6-9203-4110-93af-4a78428c672b => generated 1522 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:41:19.103341 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.policy [None req-e5d180f2-0ffb-4d8b-b0fb-91e1cd51a682 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f8725c4d05b4dd79b05b43fba15980b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0b35c2038f6b4d0a9012b004f1416c00', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125640) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:41:19.112531 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: INFO nova.api.openstack.requestlog [None req-e5d180f2-0ffb-4d8b-b0fb-91e1cd51a682 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] 10.222.0.26 "GET /compute/v2.1/servers/dc8b8a6e-eab7-4f45-8f5d-916c52eec398" status: 200 len: 1365 microversion: 2.1 time: 0.175491 Oct 19 00:41:19.113113 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: [pid: 125640|app: 0|req: 50/102] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:41:18 2022] GET /compute/v2.1/servers/dc8b8a6e-eab7-4f45-8f5d-916c52eec398 => generated 1365 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 19 00:41:19.209916 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [req-e5788cf7-73f9-4518-82c1-e91a2c173f92 req-e8fbaa1b-ffe9-4116-ba03-014f27c0068c service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "dc8b8a6e-eab7-4f45-8f5d-916c52eec398", "name": "network-vif-plugged", "status": "completed", "tag": "e00fb603-2d03-421e-b1f4-e3767c67aadc"}]} {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 19 00:41:19.221156 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-e5788cf7-73f9-4518-82c1-e91a2c173f92 req-e8fbaa1b-ffe9-4116-ba03-014f27c0068c service nova] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:41:19.221845 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-e5788cf7-73f9-4518-82c1-e91a2c173f92 req-e8fbaa1b-ffe9-4116-ba03-014f27c0068c service nova] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:41:19.222114 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-e5788cf7-73f9-4518-82c1-e91a2c173f92 req-e8fbaa1b-ffe9-4116-ba03-014f27c0068c service nova] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:41:19.237631 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.compute.server_external_events [req-e5788cf7-73f9-4518-82c1-e91a2c173f92 req-e8fbaa1b-ffe9-4116-ba03-014f27c0068c service nova] Creating event network-vif-plugged:e00fb603-2d03-421e-b1f4-e3767c67aadc for instance dc8b8a6e-eab7-4f45-8f5d-916c52eec398 on hv-cinder-maste Oct 19 00:41:19.242899 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [req-e5788cf7-73f9-4518-82c1-e91a2c173f92 req-e8fbaa1b-ffe9-4116-ba03-014f27c0068c service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.036623 Oct 19 00:41:19.243661 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 53/103] 10.222.0.26 () {64 vars in 1317 bytes} [Wed Oct 19 00:41:19 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 19 00:41:19.352012 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi [None req-c7c2ea0d-9080-4759-b633-5f8c28fb36fd tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Calling method '>' {{(pid=125640) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:41:19.352647 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.compute.api [None req-c7c2ea0d-9080-4759-b633-5f8c28fb36fd tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] [instance: e8a28aad-697f-4dee-8254-6f08e9d76168] Fetching instance by UUID {{(pid=125640) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:41:19.365643 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-c7c2ea0d-9080-4759-b633-5f8c28fb36fd tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:41:19.366169 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-c7c2ea0d-9080-4759-b633-5f8c28fb36fd tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:41:19.366656 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-c7c2ea0d-9080-4759-b633-5f8c28fb36fd tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:41:19.367613 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-c7c2ea0d-9080-4759-b633-5f8c28fb36fd tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:41:19.368039 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-c7c2ea0d-9080-4759-b633-5f8c28fb36fd tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:41:19.368470 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-c7c2ea0d-9080-4759-b633-5f8c28fb36fd tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:41:19.669876 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-0a3b0700-76cd-4917-b56e-0ce82568f291 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:41:19.670733 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-0a3b0700-76cd-4917-b56e-0ce82568f291 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] [instance: b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:41:19.681147 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-0a3b0700-76cd-4917-b56e-0ce82568f291 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:41:19.682247 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-0a3b0700-76cd-4917-b56e-0ce82568f291 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:41:19.682793 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-0a3b0700-76cd-4917-b56e-0ce82568f291 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:41:19.683966 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-0a3b0700-76cd-4917-b56e-0ce82568f291 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:41:19.684751 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-0a3b0700-76cd-4917-b56e-0ce82568f291 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:41:19.685357 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-0a3b0700-76cd-4917-b56e-0ce82568f291 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:41:19.809827 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.policy [None req-c7c2ea0d-9080-4759-b633-5f8c28fb36fd tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8118f0c698aa428a93cb054abac30c2e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f249d9041c6749f58973a5b0867b07ce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125640) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:41:19.821422 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: INFO nova.api.openstack.requestlog [None req-c7c2ea0d-9080-4759-b633-5f8c28fb36fd tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] 10.222.0.26 "GET /compute/v2.1/servers/e8a28aad-697f-4dee-8254-6f08e9d76168" status: 200 len: 1314 microversion: 2.1 time: 0.474361 Oct 19 00:41:19.821991 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: [pid: 125640|app: 0|req: 51/104] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:41:19 2022] GET /compute/v2.1/servers/e8a28aad-697f-4dee-8254-6f08e9d76168 => generated 1314 bytes in 477 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:41:19.917983 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-0a3b0700-76cd-4917-b56e-0ce82568f291 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:41:19.929896 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-0a3b0700-76cd-4917-b56e-0ce82568f291 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:41:19.931361 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-0a3b0700-76cd-4917-b56e-0ce82568f291 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:41:19.934665 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-0a3b0700-76cd-4917-b56e-0ce82568f291 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] 10.222.0.26 "GET /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2" status: 200 len: 1407 microversion: 2.20 time: 0.270127 Oct 19 00:41:19.934665 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 54/105] 10.222.0.26 () {60 vars in 1277 bytes} [Wed Oct 19 00:41:19 2022] GET /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2 => generated 1407 bytes in 272 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 19 00:41:20.060562 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi [None req-7bbec071-b380-460c-8103-b93f774140af tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Calling method '>' {{(pid=125640) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:41:20.061176 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.compute.api [None req-7bbec071-b380-460c-8103-b93f774140af tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] [instance: ab4c73d6-9203-4110-93af-4a78428c672b] Fetching instance by UUID {{(pid=125640) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:41:20.070421 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-7bbec071-b380-460c-8103-b93f774140af tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:41:20.071173 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-7bbec071-b380-460c-8103-b93f774140af tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:41:20.072305 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-7bbec071-b380-460c-8103-b93f774140af tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:41:20.073086 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-7bbec071-b380-460c-8103-b93f774140af tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:41:20.073677 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-7bbec071-b380-460c-8103-b93f774140af tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:41:20.074256 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-7bbec071-b380-460c-8103-b93f774140af tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:41:20.136889 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-46b8bfe8-1431-451f-af2c-3b683f6dc719 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:41:20.137501 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-46b8bfe8-1431-451f-af2c-3b683f6dc719 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] [instance: dc8b8a6e-eab7-4f45-8f5d-916c52eec398] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:41:20.144938 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-46b8bfe8-1431-451f-af2c-3b683f6dc719 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:41:20.145439 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-46b8bfe8-1431-451f-af2c-3b683f6dc719 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:41:20.145991 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-46b8bfe8-1431-451f-af2c-3b683f6dc719 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:41:20.152648 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-46b8bfe8-1431-451f-af2c-3b683f6dc719 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:41:20.153607 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-46b8bfe8-1431-451f-af2c-3b683f6dc719 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:41:20.155293 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-46b8bfe8-1431-451f-af2c-3b683f6dc719 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:41:20.244924 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.policy [None req-7bbec071-b380-460c-8103-b93f774140af tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd3f0d615187d4779bdd7da8c9f24c796', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0151d58abd047a692f5cc4e7d05bea5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125640) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:41:20.256686 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: INFO nova.api.openstack.requestlog [None req-7bbec071-b380-460c-8103-b93f774140af tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] 10.222.0.26 "GET /compute/v2.1/servers/ab4c73d6-9203-4110-93af-4a78428c672b" status: 200 len: 1522 microversion: 2.1 time: 0.201989 Oct 19 00:41:20.257720 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: [pid: 125640|app: 0|req: 52/106] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:41:20 2022] GET /compute/v2.1/servers/ab4c73d6-9203-4110-93af-4a78428c672b => generated 1522 bytes in 204 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:41:20.331933 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-46b8bfe8-1431-451f-af2c-3b683f6dc719 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f8725c4d05b4dd79b05b43fba15980b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0b35c2038f6b4d0a9012b004f1416c00', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:41:20.347492 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-46b8bfe8-1431-451f-af2c-3b683f6dc719 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] 10.222.0.26 "GET /compute/v2.1/servers/dc8b8a6e-eab7-4f45-8f5d-916c52eec398" status: 200 len: 1365 microversion: 2.1 time: 0.214272 Oct 19 00:41:20.349072 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 55/107] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:41:20 2022] GET /compute/v2.1/servers/dc8b8a6e-eab7-4f45-8f5d-916c52eec398 => generated 1365 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 19 00:41:20.846730 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi [None req-cf16dfad-2415-48b8-905f-0d6a45e96e29 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Calling method '>' {{(pid=125640) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:41:20.847349 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.compute.api [None req-cf16dfad-2415-48b8-905f-0d6a45e96e29 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] [instance: e8a28aad-697f-4dee-8254-6f08e9d76168] Fetching instance by UUID {{(pid=125640) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:41:20.854853 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-cf16dfad-2415-48b8-905f-0d6a45e96e29 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:41:20.855347 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-cf16dfad-2415-48b8-905f-0d6a45e96e29 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:41:20.855784 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-cf16dfad-2415-48b8-905f-0d6a45e96e29 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:41:20.856491 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-cf16dfad-2415-48b8-905f-0d6a45e96e29 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:41:20.856951 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-cf16dfad-2415-48b8-905f-0d6a45e96e29 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:41:20.857376 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-cf16dfad-2415-48b8-905f-0d6a45e96e29 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:41:20.959238 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-219571da-a2f6-48c0-8420-5d70934d2495 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:41:20.960394 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-219571da-a2f6-48c0-8420-5d70934d2495 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] [instance: b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:41:20.966537 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-219571da-a2f6-48c0-8420-5d70934d2495 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:41:20.967098 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-219571da-a2f6-48c0-8420-5d70934d2495 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:41:20.967729 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-219571da-a2f6-48c0-8420-5d70934d2495 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:41:20.968663 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-219571da-a2f6-48c0-8420-5d70934d2495 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:41:20.969179 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-219571da-a2f6-48c0-8420-5d70934d2495 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:41:20.969602 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-219571da-a2f6-48c0-8420-5d70934d2495 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:41:21.018246 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.policy [None req-cf16dfad-2415-48b8-905f-0d6a45e96e29 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8118f0c698aa428a93cb054abac30c2e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f249d9041c6749f58973a5b0867b07ce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125640) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:41:21.027085 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: INFO nova.api.openstack.requestlog [None req-cf16dfad-2415-48b8-905f-0d6a45e96e29 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] 10.222.0.26 "GET /compute/v2.1/servers/e8a28aad-697f-4dee-8254-6f08e9d76168" status: 200 len: 1314 microversion: 2.1 time: 0.184817 Oct 19 00:41:21.027881 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: [pid: 125640|app: 0|req: 53/108] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:41:20 2022] GET /compute/v2.1/servers/e8a28aad-697f-4dee-8254-6f08e9d76168 => generated 1314 bytes in 186 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:41:21.126823 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-219571da-a2f6-48c0-8420-5d70934d2495 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:41:21.140607 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-219571da-a2f6-48c0-8420-5d70934d2495 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:41:21.143287 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-219571da-a2f6-48c0-8420-5d70934d2495 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:41:21.144542 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-219571da-a2f6-48c0-8420-5d70934d2495 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] 10.222.0.26 "GET /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2" status: 200 len: 1407 microversion: 2.20 time: 0.190123 Oct 19 00:41:21.145210 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 56/109] 10.222.0.26 () {60 vars in 1277 bytes} [Wed Oct 19 00:41:20 2022] GET /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2 => generated 1407 bytes in 191 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 19 00:41:21.281307 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi [None req-62bb5b8e-dc72-43f3-a7c1-2831091029c8 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Calling method '>' {{(pid=125640) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:41:21.281915 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.compute.api [None req-62bb5b8e-dc72-43f3-a7c1-2831091029c8 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] [instance: ab4c73d6-9203-4110-93af-4a78428c672b] Fetching instance by UUID {{(pid=125640) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:41:21.289230 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-62bb5b8e-dc72-43f3-a7c1-2831091029c8 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:41:21.289709 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-62bb5b8e-dc72-43f3-a7c1-2831091029c8 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:41:21.290138 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-62bb5b8e-dc72-43f3-a7c1-2831091029c8 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:41:21.290855 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-62bb5b8e-dc72-43f3-a7c1-2831091029c8 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:41:21.291307 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-62bb5b8e-dc72-43f3-a7c1-2831091029c8 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:41:21.291721 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-62bb5b8e-dc72-43f3-a7c1-2831091029c8 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:41:21.377574 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-d18d09e8-6cdd-462a-b961-1f9bebb7d72f tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:41:21.379101 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-d18d09e8-6cdd-462a-b961-1f9bebb7d72f tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] [instance: dc8b8a6e-eab7-4f45-8f5d-916c52eec398] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:41:21.388172 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d18d09e8-6cdd-462a-b961-1f9bebb7d72f tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:41:21.390455 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d18d09e8-6cdd-462a-b961-1f9bebb7d72f tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:41:21.392057 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d18d09e8-6cdd-462a-b961-1f9bebb7d72f tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:41:21.393097 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d18d09e8-6cdd-462a-b961-1f9bebb7d72f tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:41:21.393935 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d18d09e8-6cdd-462a-b961-1f9bebb7d72f tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:41:21.395190 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d18d09e8-6cdd-462a-b961-1f9bebb7d72f tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:41:21.433712 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.policy [None req-62bb5b8e-dc72-43f3-a7c1-2831091029c8 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd3f0d615187d4779bdd7da8c9f24c796', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0151d58abd047a692f5cc4e7d05bea5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125640) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:41:21.452705 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: INFO nova.api.openstack.requestlog [None req-62bb5b8e-dc72-43f3-a7c1-2831091029c8 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] 10.222.0.26 "GET /compute/v2.1/servers/ab4c73d6-9203-4110-93af-4a78428c672b" status: 200 len: 1522 microversion: 2.1 time: 0.175093 Oct 19 00:41:21.458883 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: [pid: 125640|app: 0|req: 54/110] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:41:21 2022] GET /compute/v2.1/servers/ab4c73d6-9203-4110-93af-4a78428c672b => generated 1522 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:41:21.574797 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-d18d09e8-6cdd-462a-b961-1f9bebb7d72f tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f8725c4d05b4dd79b05b43fba15980b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0b35c2038f6b4d0a9012b004f1416c00', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:41:21.586077 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-d18d09e8-6cdd-462a-b961-1f9bebb7d72f tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] 10.222.0.26 "GET /compute/v2.1/servers/dc8b8a6e-eab7-4f45-8f5d-916c52eec398" status: 200 len: 1365 microversion: 2.1 time: 0.215360 Oct 19 00:41:21.586673 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 57/111] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:41:21 2022] GET /compute/v2.1/servers/dc8b8a6e-eab7-4f45-8f5d-916c52eec398 => generated 1365 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:41:22.051954 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi [None req-051066ff-1ddc-40a2-90b1-6d2f4ced1589 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Calling method '>' {{(pid=125640) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:41:22.052584 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.compute.api [None req-051066ff-1ddc-40a2-90b1-6d2f4ced1589 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] [instance: e8a28aad-697f-4dee-8254-6f08e9d76168] Fetching instance by UUID {{(pid=125640) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:41:22.066943 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-051066ff-1ddc-40a2-90b1-6d2f4ced1589 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:41:22.068826 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-051066ff-1ddc-40a2-90b1-6d2f4ced1589 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:41:22.069435 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-051066ff-1ddc-40a2-90b1-6d2f4ced1589 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:41:22.072088 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-051066ff-1ddc-40a2-90b1-6d2f4ced1589 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:41:22.073468 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-051066ff-1ddc-40a2-90b1-6d2f4ced1589 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:41:22.074433 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-051066ff-1ddc-40a2-90b1-6d2f4ced1589 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:41:22.170067 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-fbeb2359-0777-4436-905d-5834d8b8e01a tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:41:22.170708 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-fbeb2359-0777-4436-905d-5834d8b8e01a tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] [instance: b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:41:22.182992 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-fbeb2359-0777-4436-905d-5834d8b8e01a tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:41:22.183490 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-fbeb2359-0777-4436-905d-5834d8b8e01a tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:41:22.183918 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-fbeb2359-0777-4436-905d-5834d8b8e01a tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:41:22.184623 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-fbeb2359-0777-4436-905d-5834d8b8e01a tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:41:22.185073 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-fbeb2359-0777-4436-905d-5834d8b8e01a tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:41:22.185493 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-fbeb2359-0777-4436-905d-5834d8b8e01a tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:41:22.285181 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.policy [None req-051066ff-1ddc-40a2-90b1-6d2f4ced1589 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8118f0c698aa428a93cb054abac30c2e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f249d9041c6749f58973a5b0867b07ce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125640) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:41:22.293759 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: INFO nova.api.openstack.requestlog [None req-051066ff-1ddc-40a2-90b1-6d2f4ced1589 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] 10.222.0.26 "GET /compute/v2.1/servers/e8a28aad-697f-4dee-8254-6f08e9d76168" status: 200 len: 1502 microversion: 2.1 time: 0.247714 Oct 19 00:41:22.296180 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: [pid: 125640|app: 0|req: 55/112] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:41:22 2022] GET /compute/v2.1/servers/e8a28aad-697f-4dee-8254-6f08e9d76168 => generated 1502 bytes in 251 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:41:22.411316 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-fbeb2359-0777-4436-905d-5834d8b8e01a tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:41:22.423461 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-fbeb2359-0777-4436-905d-5834d8b8e01a tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:41:22.425602 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-fbeb2359-0777-4436-905d-5834d8b8e01a tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:41:22.427067 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-fbeb2359-0777-4436-905d-5834d8b8e01a tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] 10.222.0.26 "GET /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2" status: 200 len: 1598 microversion: 2.20 time: 0.260852 Oct 19 00:41:22.428147 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 58/113] 10.222.0.26 () {60 vars in 1277 bytes} [Wed Oct 19 00:41:22 2022] GET /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2 => generated 1598 bytes in 262 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 19 00:41:22.455663 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi [None req-360ce053-3312-4a33-bf58-66fcbea9123c tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Calling method '>' {{(pid=125640) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:41:22.456154 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.compute.api [None req-360ce053-3312-4a33-bf58-66fcbea9123c tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] [instance: b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2] Fetching instance by UUID {{(pid=125640) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:41:22.463580 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-360ce053-3312-4a33-bf58-66fcbea9123c tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:41:22.464438 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-360ce053-3312-4a33-bf58-66fcbea9123c tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:41:22.465041 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-360ce053-3312-4a33-bf58-66fcbea9123c tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:41:22.466230 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-360ce053-3312-4a33-bf58-66fcbea9123c tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:41:22.466933 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-360ce053-3312-4a33-bf58-66fcbea9123c tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:41:22.467753 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-360ce053-3312-4a33-bf58-66fcbea9123c tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:41:22.484142 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-07009843-61a3-414b-8750-94ff6d12cd3a tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:41:22.484809 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-07009843-61a3-414b-8750-94ff6d12cd3a tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] [instance: ab4c73d6-9203-4110-93af-4a78428c672b] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:41:22.498132 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-07009843-61a3-414b-8750-94ff6d12cd3a tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:41:22.499928 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-07009843-61a3-414b-8750-94ff6d12cd3a tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:41:22.503720 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-07009843-61a3-414b-8750-94ff6d12cd3a tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:41:22.504689 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-07009843-61a3-414b-8750-94ff6d12cd3a tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:41:22.505304 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-07009843-61a3-414b-8750-94ff6d12cd3a tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:41:22.506896 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-07009843-61a3-414b-8750-94ff6d12cd3a tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:41:22.597227 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: INFO nova.api.openstack.requestlog [None req-360ce053-3312-4a33-bf58-66fcbea9123c tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] 10.222.0.26 "GET /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2/os-interface" status: 200 len: 284 microversion: 2.20 time: 0.146296 Oct 19 00:41:22.598292 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: [pid: 125640|app: 0|req: 56/114] 10.222.0.26 () {60 vars in 1316 bytes} [Wed Oct 19 00:41:22 2022] GET /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2/os-interface => generated 284 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 19 00:41:22.612987 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi [None req-f9168abc-4f21-4b4d-93cf-e80df1da993a tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Calling method '>' {{(pid=125640) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:41:22.613992 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.compute.api [None req-f9168abc-4f21-4b4d-93cf-e80df1da993a tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] [instance: dc8b8a6e-eab7-4f45-8f5d-916c52eec398] Fetching instance by UUID {{(pid=125640) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:41:22.625267 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-f9168abc-4f21-4b4d-93cf-e80df1da993a tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:41:22.627975 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-f9168abc-4f21-4b4d-93cf-e80df1da993a tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:41:22.628617 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-f9168abc-4f21-4b4d-93cf-e80df1da993a tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:41:22.631133 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-f9168abc-4f21-4b4d-93cf-e80df1da993a tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:41:22.635471 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-f9168abc-4f21-4b4d-93cf-e80df1da993a tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:41:22.636084 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-f9168abc-4f21-4b4d-93cf-e80df1da993a tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:41:22.743214 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-07009843-61a3-414b-8750-94ff6d12cd3a tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd3f0d615187d4779bdd7da8c9f24c796', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0151d58abd047a692f5cc4e7d05bea5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:41:22.757005 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-07009843-61a3-414b-8750-94ff6d12cd3a tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] 10.222.0.26 "GET /compute/v2.1/servers/ab4c73d6-9203-4110-93af-4a78428c672b" status: 200 len: 1522 microversion: 2.1 time: 0.277750 Oct 19 00:41:22.758204 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 59/115] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:41:22 2022] GET /compute/v2.1/servers/ab4c73d6-9203-4110-93af-4a78428c672b => generated 1522 bytes in 281 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:41:23.326676 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.policy [None req-f9168abc-4f21-4b4d-93cf-e80df1da993a tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f8725c4d05b4dd79b05b43fba15980b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0b35c2038f6b4d0a9012b004f1416c00', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125640) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:41:23.340428 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: INFO nova.api.openstack.requestlog [None req-f9168abc-4f21-4b4d-93cf-e80df1da993a tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] 10.222.0.26 "GET /compute/v2.1/servers/dc8b8a6e-eab7-4f45-8f5d-916c52eec398" status: 200 len: 1365 microversion: 2.1 time: 0.732064 Oct 19 00:41:23.355202 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: [pid: 125640|app: 0|req: 57/116] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:41:22 2022] GET /compute/v2.1/servers/dc8b8a6e-eab7-4f45-8f5d-916c52eec398 => generated 1365 bytes in 747 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:41:23.781379 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-16d83291-ead0-48cc-8908-f6737cdbdeaf tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:41:23.782061 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-16d83291-ead0-48cc-8908-f6737cdbdeaf tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] [instance: ab4c73d6-9203-4110-93af-4a78428c672b] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:41:23.791486 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-16d83291-ead0-48cc-8908-f6737cdbdeaf tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:41:23.792571 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-16d83291-ead0-48cc-8908-f6737cdbdeaf tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:41:23.794306 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-16d83291-ead0-48cc-8908-f6737cdbdeaf tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:41:23.796485 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-16d83291-ead0-48cc-8908-f6737cdbdeaf tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:41:23.797185 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-16d83291-ead0-48cc-8908-f6737cdbdeaf tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:41:23.799160 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-16d83291-ead0-48cc-8908-f6737cdbdeaf tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:41:23.848039 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi [None req-54f840e2-be95-45a5-ad5d-03b8e514d3a8 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "900d4e0e-1bf1-432a-bd4e-9ea80a3bd45c"}} {{(pid=125640) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 19 00:41:23.851895 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.compute.api [None req-54f840e2-be95-45a5-ad5d-03b8e514d3a8 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] [instance: e8a28aad-697f-4dee-8254-6f08e9d76168] Fetching instance by UUID {{(pid=125640) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:41:23.873141 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-54f840e2-be95-45a5-ad5d-03b8e514d3a8 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:41:23.873652 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-54f840e2-be95-45a5-ad5d-03b8e514d3a8 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:41:23.874095 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-54f840e2-be95-45a5-ad5d-03b8e514d3a8 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:41:23.882056 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-54f840e2-be95-45a5-ad5d-03b8e514d3a8 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:41:23.885138 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-54f840e2-be95-45a5-ad5d-03b8e514d3a8 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.009s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:41:23.887849 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-54f840e2-be95-45a5-ad5d-03b8e514d3a8 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:41:24.130593 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-16d83291-ead0-48cc-8908-f6737cdbdeaf tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd3f0d615187d4779bdd7da8c9f24c796', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0151d58abd047a692f5cc4e7d05bea5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:41:24.138710 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-16d83291-ead0-48cc-8908-f6737cdbdeaf tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] 10.222.0.26 "GET /compute/v2.1/servers/ab4c73d6-9203-4110-93af-4a78428c672b" status: 200 len: 1522 microversion: 2.1 time: 0.361127 Oct 19 00:41:24.139300 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 60/117] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:41:23 2022] GET /compute/v2.1/servers/ab4c73d6-9203-4110-93af-4a78428c672b => generated 1522 bytes in 362 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:41:24.372636 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: INFO nova.api.openstack.requestlog [None req-54f840e2-be95-45a5-ad5d-03b8e514d3a8 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] 10.222.0.26 "POST /compute/v2.1/servers/e8a28aad-697f-4dee-8254-6f08e9d76168/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.529896 Oct 19 00:41:24.373435 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: [pid: 125640|app: 0|req: 58/118] 10.222.0.26 () {60 vars in 1323 bytes} [Wed Oct 19 00:41:23 2022] POST /compute/v2.1/servers/e8a28aad-697f-4dee-8254-6f08e9d76168/os-volume_attachments => generated 194 bytes in 532 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 19 00:41:24.373660 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-58368b9a-8b73-49ac-b30d-438e057419fd tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:41:24.374449 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-58368b9a-8b73-49ac-b30d-438e057419fd tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] [instance: dc8b8a6e-eab7-4f45-8f5d-916c52eec398] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:41:24.390146 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-58368b9a-8b73-49ac-b30d-438e057419fd tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:41:24.390691 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-58368b9a-8b73-49ac-b30d-438e057419fd tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:41:24.391136 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-58368b9a-8b73-49ac-b30d-438e057419fd tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:41:24.392528 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-58368b9a-8b73-49ac-b30d-438e057419fd tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:41:24.393272 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-58368b9a-8b73-49ac-b30d-438e057419fd tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:41:24.393710 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-58368b9a-8b73-49ac-b30d-438e057419fd tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:41:24.428531 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG dbcounter [-] [125641] Writing DB stats nova_cell1:SELECT=100 {{(pid=125641) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 00:41:24.701561 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-58368b9a-8b73-49ac-b30d-438e057419fd tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f8725c4d05b4dd79b05b43fba15980b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0b35c2038f6b4d0a9012b004f1416c00', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:41:24.711901 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-58368b9a-8b73-49ac-b30d-438e057419fd tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] 10.222.0.26 "GET /compute/v2.1/servers/dc8b8a6e-eab7-4f45-8f5d-916c52eec398" status: 200 len: 1553 microversion: 2.1 time: 0.342341 Oct 19 00:41:24.712475 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 61/119] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:41:24 2022] GET /compute/v2.1/servers/dc8b8a6e-eab7-4f45-8f5d-916c52eec398 => generated 1553 bytes in 344 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:41:24.740450 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi [None req-6f1804f8-8807-4789-8441-5f62d81a89fb tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Calling method '>' {{(pid=125640) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:41:24.741408 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.compute.api [None req-6f1804f8-8807-4789-8441-5f62d81a89fb tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] [instance: dc8b8a6e-eab7-4f45-8f5d-916c52eec398] Fetching instance by UUID {{(pid=125640) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:41:24.749449 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-6f1804f8-8807-4789-8441-5f62d81a89fb tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:41:24.750289 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-6f1804f8-8807-4789-8441-5f62d81a89fb tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:41:24.751894 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-6f1804f8-8807-4789-8441-5f62d81a89fb tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:41:24.753192 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-6f1804f8-8807-4789-8441-5f62d81a89fb tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:41:24.753997 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-6f1804f8-8807-4789-8441-5f62d81a89fb tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:41:24.754788 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-6f1804f8-8807-4789-8441-5f62d81a89fb tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:41:25.165006 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-73935d30-1646-46a8-ab22-b44a5d781415 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:41:25.166658 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-73935d30-1646-46a8-ab22-b44a5d781415 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] [instance: ab4c73d6-9203-4110-93af-4a78428c672b] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:41:25.182387 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-73935d30-1646-46a8-ab22-b44a5d781415 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:41:25.186008 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: INFO nova.api.openstack.requestlog [None req-6f1804f8-8807-4789-8441-5f62d81a89fb tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] 10.222.0.26 "GET /compute/v2.1/servers/dc8b8a6e-eab7-4f45-8f5d-916c52eec398/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.450510 Oct 19 00:41:25.190800 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: [pid: 125640|app: 0|req: 59/120] 10.222.0.26 () {58 vars in 1275 bytes} [Wed Oct 19 00:41:24 2022] GET /compute/v2.1/servers/dc8b8a6e-eab7-4f45-8f5d-916c52eec398/os-interface => generated 284 bytes in 453 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 19 00:41:25.192125 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi [req-40a840b8-6fc1-41e6-aa70-734437d68400 req-6eb0eddf-e4aa-4a92-bc72-292bac156b5f service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2", "tag": "6aea38b4-b7a0-4e8e-82dc-deb19c260092"}]} {{(pid=125640) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 19 00:41:25.195944 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-73935d30-1646-46a8-ab22-b44a5d781415 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.009s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:41:25.199116 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-73935d30-1646-46a8-ab22-b44a5d781415 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:41:25.202946 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-73935d30-1646-46a8-ab22-b44a5d781415 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:41:25.204803 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-73935d30-1646-46a8-ab22-b44a5d781415 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:41:25.209747 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-73935d30-1646-46a8-ab22-b44a5d781415 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:41:25.216301 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [req-40a840b8-6fc1-41e6-aa70-734437d68400 req-6eb0eddf-e4aa-4a92-bc72-292bac156b5f service nova] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:41:25.218023 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [req-40a840b8-6fc1-41e6-aa70-734437d68400 req-6eb0eddf-e4aa-4a92-bc72-292bac156b5f service nova] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:41:25.220958 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [req-40a840b8-6fc1-41e6-aa70-734437d68400 req-6eb0eddf-e4aa-4a92-bc72-292bac156b5f service nova] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:41:25.242731 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: INFO nova.api.openstack.compute.server_external_events [req-40a840b8-6fc1-41e6-aa70-734437d68400 req-6eb0eddf-e4aa-4a92-bc72-292bac156b5f service nova] Creating event network-changed:6aea38b4-b7a0-4e8e-82dc-deb19c260092 for instance b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2 on hv-cinder-maste Oct 19 00:41:25.258376 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: INFO nova.api.openstack.requestlog [req-40a840b8-6fc1-41e6-aa70-734437d68400 req-6eb0eddf-e4aa-4a92-bc72-292bac156b5f service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.070177 Oct 19 00:41:25.261044 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: [pid: 125640|app: 0|req: 60/121] 10.222.0.26 () {64 vars in 1317 bytes} [Wed Oct 19 00:41:25 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 74 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 19 00:41:25.940833 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-73935d30-1646-46a8-ab22-b44a5d781415 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd3f0d615187d4779bdd7da8c9f24c796', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0151d58abd047a692f5cc4e7d05bea5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:41:25.954451 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-73935d30-1646-46a8-ab22-b44a5d781415 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] 10.222.0.26 "GET /compute/v2.1/servers/ab4c73d6-9203-4110-93af-4a78428c672b" status: 200 len: 1522 microversion: 2.1 time: 0.793784 Oct 19 00:41:25.955642 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 62/122] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:41:25 2022] GET /compute/v2.1/servers/ab4c73d6-9203-4110-93af-4a78428c672b => generated 1522 bytes in 796 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 19 00:41:26.982136 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi [None req-6e6f9bed-c6dd-4df3-8319-831572286ef4 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Calling method '>' {{(pid=125640) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:41:26.983127 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.compute.api [None req-6e6f9bed-c6dd-4df3-8319-831572286ef4 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] [instance: ab4c73d6-9203-4110-93af-4a78428c672b] Fetching instance by UUID {{(pid=125640) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:41:26.998879 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-6e6f9bed-c6dd-4df3-8319-831572286ef4 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:41:26.999369 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-6e6f9bed-c6dd-4df3-8319-831572286ef4 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:41:26.999837 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-6e6f9bed-c6dd-4df3-8319-831572286ef4 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:41:27.003095 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-6e6f9bed-c6dd-4df3-8319-831572286ef4 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:41:27.005529 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-6e6f9bed-c6dd-4df3-8319-831572286ef4 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:41:27.007119 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-6e6f9bed-c6dd-4df3-8319-831572286ef4 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:41:27.185823 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.policy [None req-6e6f9bed-c6dd-4df3-8319-831572286ef4 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd3f0d615187d4779bdd7da8c9f24c796', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0151d58abd047a692f5cc4e7d05bea5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125640) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:41:27.197725 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: INFO nova.api.openstack.requestlog [None req-6e6f9bed-c6dd-4df3-8319-831572286ef4 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] 10.222.0.26 "GET /compute/v2.1/servers/ab4c73d6-9203-4110-93af-4a78428c672b" status: 200 len: 1522 microversion: 2.1 time: 0.219709 Oct 19 00:41:27.211219 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: [pid: 125640|app: 0|req: 61/123] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:41:26 2022] GET /compute/v2.1/servers/ab4c73d6-9203-4110-93af-4a78428c672b => generated 1522 bytes in 235 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:41:27.582234 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [req-d261079c-32d5-4cb4-8051-6d9d356cb3ab req-fd6357e4-5f14-4674-ace8-25173a52541b service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "dc8b8a6e-eab7-4f45-8f5d-916c52eec398", "tag": "e00fb603-2d03-421e-b1f4-e3767c67aadc"}]} {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 19 00:41:27.603553 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-d261079c-32d5-4cb4-8051-6d9d356cb3ab req-fd6357e4-5f14-4674-ace8-25173a52541b service nova] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:41:27.604081 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-d261079c-32d5-4cb4-8051-6d9d356cb3ab req-fd6357e4-5f14-4674-ace8-25173a52541b service nova] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:41:27.604516 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-d261079c-32d5-4cb4-8051-6d9d356cb3ab req-fd6357e4-5f14-4674-ace8-25173a52541b service nova] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:41:27.627376 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.compute.server_external_events [req-d261079c-32d5-4cb4-8051-6d9d356cb3ab req-fd6357e4-5f14-4674-ace8-25173a52541b service nova] Creating event network-changed:e00fb603-2d03-421e-b1f4-e3767c67aadc for instance dc8b8a6e-eab7-4f45-8f5d-916c52eec398 on hv-cinder-maste Oct 19 00:41:27.639450 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [req-d261079c-32d5-4cb4-8051-6d9d356cb3ab req-fd6357e4-5f14-4674-ace8-25173a52541b service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.061089 Oct 19 00:41:27.642989 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 63/124] 10.222.0.26 () {64 vars in 1317 bytes} [Wed Oct 19 00:41:27 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 65 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 19 00:41:28.222351 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi [None req-ba8ea7d1-086f-4d51-9c9d-cbce3e930655 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Calling method '>' {{(pid=125640) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:41:28.223976 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.compute.api [None req-ba8ea7d1-086f-4d51-9c9d-cbce3e930655 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] [instance: ab4c73d6-9203-4110-93af-4a78428c672b] Fetching instance by UUID {{(pid=125640) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:41:28.234715 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-ba8ea7d1-086f-4d51-9c9d-cbce3e930655 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:41:28.237526 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-ba8ea7d1-086f-4d51-9c9d-cbce3e930655 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:41:28.239657 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-ba8ea7d1-086f-4d51-9c9d-cbce3e930655 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:41:28.240583 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-ba8ea7d1-086f-4d51-9c9d-cbce3e930655 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:41:28.242465 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-ba8ea7d1-086f-4d51-9c9d-cbce3e930655 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:41:28.243705 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-ba8ea7d1-086f-4d51-9c9d-cbce3e930655 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:41:28.249693 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG dbcounter [-] [125640] Writing DB stats nova_cell1:SELECT=94,nova_cell1:UPDATE=4,nova_cell1:INSERT=2 {{(pid=125640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 00:41:28.590286 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.policy [None req-ba8ea7d1-086f-4d51-9c9d-cbce3e930655 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd3f0d615187d4779bdd7da8c9f24c796', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0151d58abd047a692f5cc4e7d05bea5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125640) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:41:28.605181 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: INFO nova.api.openstack.requestlog [None req-ba8ea7d1-086f-4d51-9c9d-cbce3e930655 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] 10.222.0.26 "GET /compute/v2.1/servers/ab4c73d6-9203-4110-93af-4a78428c672b" status: 200 len: 1522 microversion: 2.1 time: 0.385899 Oct 19 00:41:28.607655 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: [pid: 125640|app: 0|req: 62/125] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:41:28 2022] GET /compute/v2.1/servers/ab4c73d6-9203-4110-93af-4a78428c672b => generated 1522 bytes in 391 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:41:29.635295 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-9eba87fd-fb97-4fd1-9983-f53da56072be tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:41:29.637325 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-9eba87fd-fb97-4fd1-9983-f53da56072be tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] [instance: ab4c73d6-9203-4110-93af-4a78428c672b] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:41:29.663940 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-9eba87fd-fb97-4fd1-9983-f53da56072be tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:41:29.668088 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-9eba87fd-fb97-4fd1-9983-f53da56072be tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:41:29.668882 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-9eba87fd-fb97-4fd1-9983-f53da56072be tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:41:29.676663 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-9eba87fd-fb97-4fd1-9983-f53da56072be tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:41:29.677217 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-9eba87fd-fb97-4fd1-9983-f53da56072be tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:41:29.677771 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-9eba87fd-fb97-4fd1-9983-f53da56072be tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:41:30.274050 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-9eba87fd-fb97-4fd1-9983-f53da56072be tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd3f0d615187d4779bdd7da8c9f24c796', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0151d58abd047a692f5cc4e7d05bea5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:41:30.283756 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-9eba87fd-fb97-4fd1-9983-f53da56072be tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] 10.222.0.26 "GET /compute/v2.1/servers/ab4c73d6-9203-4110-93af-4a78428c672b" status: 200 len: 1522 microversion: 2.1 time: 0.657094 Oct 19 00:41:30.284578 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 64/126] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:41:29 2022] GET /compute/v2.1/servers/ab4c73d6-9203-4110-93af-4a78428c672b => generated 1522 bytes in 659 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:41:31.308959 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi [None req-cecb8e16-c92e-4b5f-9fd7-8faa2ebef94b tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Calling method '>' {{(pid=125640) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:41:31.309610 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.compute.api [None req-cecb8e16-c92e-4b5f-9fd7-8faa2ebef94b tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] [instance: ab4c73d6-9203-4110-93af-4a78428c672b] Fetching instance by UUID {{(pid=125640) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:41:31.317909 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-cecb8e16-c92e-4b5f-9fd7-8faa2ebef94b tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:41:31.318812 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-cecb8e16-c92e-4b5f-9fd7-8faa2ebef94b tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:41:31.319336 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-cecb8e16-c92e-4b5f-9fd7-8faa2ebef94b tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:41:31.320335 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-cecb8e16-c92e-4b5f-9fd7-8faa2ebef94b tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:41:31.321063 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-cecb8e16-c92e-4b5f-9fd7-8faa2ebef94b tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:41:31.321773 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-cecb8e16-c92e-4b5f-9fd7-8faa2ebef94b tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:41:31.505815 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.policy [None req-cecb8e16-c92e-4b5f-9fd7-8faa2ebef94b tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd3f0d615187d4779bdd7da8c9f24c796', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0151d58abd047a692f5cc4e7d05bea5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125640) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:41:31.515138 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: INFO nova.api.openstack.requestlog [None req-cecb8e16-c92e-4b5f-9fd7-8faa2ebef94b tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] 10.222.0.26 "GET /compute/v2.1/servers/ab4c73d6-9203-4110-93af-4a78428c672b" status: 200 len: 1522 microversion: 2.1 time: 0.210569 Oct 19 00:41:31.520858 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: [pid: 125640|app: 0|req: 63/127] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:41:31 2022] GET /compute/v2.1/servers/ab4c73d6-9203-4110-93af-4a78428c672b => generated 1522 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:41:32.539260 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-292ca037-9413-4142-acb5-0be0d0969ec0 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:41:32.539869 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-292ca037-9413-4142-acb5-0be0d0969ec0 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] [instance: ab4c73d6-9203-4110-93af-4a78428c672b] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:41:32.546392 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-292ca037-9413-4142-acb5-0be0d0969ec0 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:41:32.546970 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-292ca037-9413-4142-acb5-0be0d0969ec0 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:41:32.547597 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-292ca037-9413-4142-acb5-0be0d0969ec0 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:41:32.548357 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-292ca037-9413-4142-acb5-0be0d0969ec0 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:41:32.562758 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-292ca037-9413-4142-acb5-0be0d0969ec0 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:41:32.563100 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-292ca037-9413-4142-acb5-0be0d0969ec0 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.014s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:41:32.723111 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-292ca037-9413-4142-acb5-0be0d0969ec0 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd3f0d615187d4779bdd7da8c9f24c796', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0151d58abd047a692f5cc4e7d05bea5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:41:32.735936 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-292ca037-9413-4142-acb5-0be0d0969ec0 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] 10.222.0.26 "GET /compute/v2.1/servers/ab4c73d6-9203-4110-93af-4a78428c672b" status: 200 len: 1522 microversion: 2.1 time: 0.199768 Oct 19 00:41:32.736800 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 65/128] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:41:32 2022] GET /compute/v2.1/servers/ab4c73d6-9203-4110-93af-4a78428c672b => generated 1522 bytes in 202 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:41:33.762445 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi [None req-ff892e28-20af-4b3c-abd6-979f36589ab0 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Calling method '>' {{(pid=125640) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:41:33.763829 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.compute.api [None req-ff892e28-20af-4b3c-abd6-979f36589ab0 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] [instance: ab4c73d6-9203-4110-93af-4a78428c672b] Fetching instance by UUID {{(pid=125640) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:41:33.776770 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-ff892e28-20af-4b3c-abd6-979f36589ab0 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:41:33.777279 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-ff892e28-20af-4b3c-abd6-979f36589ab0 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:41:33.777733 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-ff892e28-20af-4b3c-abd6-979f36589ab0 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:41:33.778434 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-ff892e28-20af-4b3c-abd6-979f36589ab0 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:41:33.778922 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-ff892e28-20af-4b3c-abd6-979f36589ab0 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:41:33.779352 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-ff892e28-20af-4b3c-abd6-979f36589ab0 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:41:34.266353 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.policy [None req-ff892e28-20af-4b3c-abd6-979f36589ab0 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd3f0d615187d4779bdd7da8c9f24c796', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0151d58abd047a692f5cc4e7d05bea5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125640) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:41:34.281408 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: INFO nova.api.openstack.requestlog [None req-ff892e28-20af-4b3c-abd6-979f36589ab0 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] 10.222.0.26 "GET /compute/v2.1/servers/ab4c73d6-9203-4110-93af-4a78428c672b" status: 200 len: 1522 microversion: 2.1 time: 0.523576 Oct 19 00:41:34.287573 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: [pid: 125640|app: 0|req: 64/129] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:41:33 2022] GET /compute/v2.1/servers/ab4c73d6-9203-4110-93af-4a78428c672b => generated 1522 bytes in 531 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:41:34.733147 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-bcafb3c7-86a0-4d4a-82e5-d22ea86972aa tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:41:34.737243 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-bcafb3c7-86a0-4d4a-82e5-d22ea86972aa tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] [instance: e8a28aad-697f-4dee-8254-6f08e9d76168] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:41:34.764432 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-bcafb3c7-86a0-4d4a-82e5-d22ea86972aa tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:41:34.769412 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-bcafb3c7-86a0-4d4a-82e5-d22ea86972aa tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:41:34.770932 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-bcafb3c7-86a0-4d4a-82e5-d22ea86972aa tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:41:34.772489 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-bcafb3c7-86a0-4d4a-82e5-d22ea86972aa tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:41:34.775655 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-bcafb3c7-86a0-4d4a-82e5-d22ea86972aa tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:41:34.776778 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-bcafb3c7-86a0-4d4a-82e5-d22ea86972aa tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:41:34.827470 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-bcafb3c7-86a0-4d4a-82e5-d22ea86972aa tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] [instance: e8a28aad-697f-4dee-8254-6f08e9d76168] Going to try to terminate instance {{(pid=125641) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 19 00:41:35.078044 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-bcafb3c7-86a0-4d4a-82e5-d22ea86972aa tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] 10.222.0.26 "DELETE /compute/v2.1/servers/e8a28aad-697f-4dee-8254-6f08e9d76168" status: 204 len: 0 microversion: 2.1 time: 0.348021 Oct 19 00:41:35.079409 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 66/130] 10.222.0.26 () {58 vars in 1239 bytes} [Wed Oct 19 00:41:34 2022] DELETE /compute/v2.1/servers/e8a28aad-697f-4dee-8254-6f08e9d76168 => generated 0 bytes in 351 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 19 00:41:35.095697 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi [None req-72bdfb5b-226b-4d46-9d1d-9262ae856992 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Calling method '>' {{(pid=125640) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:41:35.097647 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.compute.api [None req-72bdfb5b-226b-4d46-9d1d-9262ae856992 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] [instance: e8a28aad-697f-4dee-8254-6f08e9d76168] Fetching instance by UUID {{(pid=125640) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:41:35.116848 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG dbcounter [-] [125640] Writing DB stats nova_api:SELECT=77 {{(pid=125640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 00:41:35.128312 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-72bdfb5b-226b-4d46-9d1d-9262ae856992 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:41:35.130899 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-72bdfb5b-226b-4d46-9d1d-9262ae856992 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:41:35.131751 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-72bdfb5b-226b-4d46-9d1d-9262ae856992 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:41:35.134787 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-72bdfb5b-226b-4d46-9d1d-9262ae856992 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:41:35.136569 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-72bdfb5b-226b-4d46-9d1d-9262ae856992 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:41:35.137327 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-72bdfb5b-226b-4d46-9d1d-9262ae856992 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:41:35.319721 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-304b2f5a-cfb0-4251-bed8-b15c404f9c61 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:41:35.320424 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-304b2f5a-cfb0-4251-bed8-b15c404f9c61 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] [instance: ab4c73d6-9203-4110-93af-4a78428c672b] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:41:35.333881 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-304b2f5a-cfb0-4251-bed8-b15c404f9c61 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:41:35.334378 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-304b2f5a-cfb0-4251-bed8-b15c404f9c61 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:41:35.334834 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-304b2f5a-cfb0-4251-bed8-b15c404f9c61 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:41:35.339094 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-304b2f5a-cfb0-4251-bed8-b15c404f9c61 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:41:35.339842 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-304b2f5a-cfb0-4251-bed8-b15c404f9c61 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:41:35.340457 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-304b2f5a-cfb0-4251-bed8-b15c404f9c61 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:41:35.403235 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.policy [None req-72bdfb5b-226b-4d46-9d1d-9262ae856992 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8118f0c698aa428a93cb054abac30c2e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f249d9041c6749f58973a5b0867b07ce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125640) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:41:35.420023 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: INFO nova.api.openstack.requestlog [None req-72bdfb5b-226b-4d46-9d1d-9262ae856992 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] 10.222.0.26 "GET /compute/v2.1/servers/e8a28aad-697f-4dee-8254-6f08e9d76168" status: 200 len: 1554 microversion: 2.1 time: 0.334844 Oct 19 00:41:35.420634 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: [pid: 125640|app: 0|req: 65/131] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:41:35 2022] GET /compute/v2.1/servers/e8a28aad-697f-4dee-8254-6f08e9d76168 => generated 1554 bytes in 337 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:41:35.676150 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-304b2f5a-cfb0-4251-bed8-b15c404f9c61 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd3f0d615187d4779bdd7da8c9f24c796', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0151d58abd047a692f5cc4e7d05bea5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:41:35.690221 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-304b2f5a-cfb0-4251-bed8-b15c404f9c61 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] 10.222.0.26 "GET /compute/v2.1/servers/ab4c73d6-9203-4110-93af-4a78428c672b" status: 200 len: 1522 microversion: 2.1 time: 0.376035 Oct 19 00:41:35.691179 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 67/132] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:41:35 2022] GET /compute/v2.1/servers/ab4c73d6-9203-4110-93af-4a78428c672b => generated 1522 bytes in 379 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:41:36.448270 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi [None req-949b92b7-10a3-4831-9d39-0c89069fb17c tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Calling method '>' {{(pid=125640) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:41:36.448896 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.compute.api [None req-949b92b7-10a3-4831-9d39-0c89069fb17c tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] [instance: e8a28aad-697f-4dee-8254-6f08e9d76168] Fetching instance by UUID {{(pid=125640) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:41:36.457270 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-949b92b7-10a3-4831-9d39-0c89069fb17c tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:41:36.458145 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-949b92b7-10a3-4831-9d39-0c89069fb17c tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:41:36.459039 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-949b92b7-10a3-4831-9d39-0c89069fb17c tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:41:36.459949 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-949b92b7-10a3-4831-9d39-0c89069fb17c tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:41:36.460611 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-949b92b7-10a3-4831-9d39-0c89069fb17c tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:41:36.461269 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-949b92b7-10a3-4831-9d39-0c89069fb17c tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:41:36.645179 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.policy [None req-949b92b7-10a3-4831-9d39-0c89069fb17c tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8118f0c698aa428a93cb054abac30c2e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f249d9041c6749f58973a5b0867b07ce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125640) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:41:36.656167 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: INFO nova.api.openstack.requestlog [None req-949b92b7-10a3-4831-9d39-0c89069fb17c tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] 10.222.0.26 "GET /compute/v2.1/servers/e8a28aad-697f-4dee-8254-6f08e9d76168" status: 200 len: 1554 microversion: 2.1 time: 0.212166 Oct 19 00:41:36.657378 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: [pid: 125640|app: 0|req: 66/133] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:41:36 2022] GET /compute/v2.1/servers/e8a28aad-697f-4dee-8254-6f08e9d76168 => generated 1554 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 19 00:41:36.714878 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-258ad698-f6f2-48f0-b8e1-8b7aac951327 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:41:36.716102 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-258ad698-f6f2-48f0-b8e1-8b7aac951327 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] [instance: ab4c73d6-9203-4110-93af-4a78428c672b] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:41:36.723019 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG dbcounter [-] [125641] Writing DB stats nova_api:SELECT=80,nova_api:UPDATE=1 {{(pid=125641) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 00:41:36.724659 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-258ad698-f6f2-48f0-b8e1-8b7aac951327 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:41:36.725136 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-258ad698-f6f2-48f0-b8e1-8b7aac951327 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:41:36.725717 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-258ad698-f6f2-48f0-b8e1-8b7aac951327 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:41:36.726640 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-258ad698-f6f2-48f0-b8e1-8b7aac951327 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:41:36.727096 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-258ad698-f6f2-48f0-b8e1-8b7aac951327 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:41:36.727520 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-258ad698-f6f2-48f0-b8e1-8b7aac951327 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:41:37.173231 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-258ad698-f6f2-48f0-b8e1-8b7aac951327 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd3f0d615187d4779bdd7da8c9f24c796', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0151d58abd047a692f5cc4e7d05bea5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:41:37.186459 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-258ad698-f6f2-48f0-b8e1-8b7aac951327 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] 10.222.0.26 "GET /compute/v2.1/servers/ab4c73d6-9203-4110-93af-4a78428c672b" status: 200 len: 1522 microversion: 2.1 time: 0.475621 Oct 19 00:41:37.187807 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 68/134] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:41:36 2022] GET /compute/v2.1/servers/ab4c73d6-9203-4110-93af-4a78428c672b => generated 1522 bytes in 478 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:41:37.702006 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi [None req-65f5e8da-3a20-4514-8009-1ccbfff64874 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Calling method '>' {{(pid=125640) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:41:37.702769 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.compute.api [None req-65f5e8da-3a20-4514-8009-1ccbfff64874 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] [instance: e8a28aad-697f-4dee-8254-6f08e9d76168] Fetching instance by UUID {{(pid=125640) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:41:37.716332 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-65f5e8da-3a20-4514-8009-1ccbfff64874 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:41:37.716832 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-65f5e8da-3a20-4514-8009-1ccbfff64874 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:41:37.717267 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-65f5e8da-3a20-4514-8009-1ccbfff64874 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:41:37.722059 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-65f5e8da-3a20-4514-8009-1ccbfff64874 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:41:37.722684 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-65f5e8da-3a20-4514-8009-1ccbfff64874 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:41:37.723133 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-65f5e8da-3a20-4514-8009-1ccbfff64874 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:41:38.215858 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-3fc3e904-2b5b-4373-81bf-7f5de7c77be6 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:41:38.223092 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-3fc3e904-2b5b-4373-81bf-7f5de7c77be6 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] [instance: ab4c73d6-9203-4110-93af-4a78428c672b] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:41:38.239089 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-3fc3e904-2b5b-4373-81bf-7f5de7c77be6 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:41:38.244084 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-3fc3e904-2b5b-4373-81bf-7f5de7c77be6 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:41:38.246443 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-3fc3e904-2b5b-4373-81bf-7f5de7c77be6 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:41:38.248226 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-3fc3e904-2b5b-4373-81bf-7f5de7c77be6 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:41:38.251996 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-3fc3e904-2b5b-4373-81bf-7f5de7c77be6 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:41:38.252729 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-3fc3e904-2b5b-4373-81bf-7f5de7c77be6 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:41:38.460769 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.policy [None req-65f5e8da-3a20-4514-8009-1ccbfff64874 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8118f0c698aa428a93cb054abac30c2e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f249d9041c6749f58973a5b0867b07ce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125640) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:41:38.492401 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: INFO nova.api.openstack.requestlog [None req-65f5e8da-3a20-4514-8009-1ccbfff64874 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] 10.222.0.26 "GET /compute/v2.1/servers/e8a28aad-697f-4dee-8254-6f08e9d76168" status: 200 len: 1554 microversion: 2.1 time: 0.799227 Oct 19 00:41:38.493506 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: [pid: 125640|app: 0|req: 67/135] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:41:37 2022] GET /compute/v2.1/servers/e8a28aad-697f-4dee-8254-6f08e9d76168 => generated 1554 bytes in 805 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:41:38.619524 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-3fc3e904-2b5b-4373-81bf-7f5de7c77be6 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd3f0d615187d4779bdd7da8c9f24c796', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0151d58abd047a692f5cc4e7d05bea5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:41:38.634628 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-3fc3e904-2b5b-4373-81bf-7f5de7c77be6 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] 10.222.0.26 "GET /compute/v2.1/servers/ab4c73d6-9203-4110-93af-4a78428c672b" status: 200 len: 1522 microversion: 2.1 time: 0.422939 Oct 19 00:41:38.651615 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 69/136] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:41:38 2022] GET /compute/v2.1/servers/ab4c73d6-9203-4110-93af-4a78428c672b => generated 1522 bytes in 442 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:41:38.791852 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi [req-159c42e2-780b-4f7e-a9c4-5ac1c5d7eddc req-1edeb3fc-540d-4d13-90be-cf99dae543bf service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "e8a28aad-697f-4dee-8254-6f08e9d76168", "name": "network-vif-deleted", "tag": "522599ca-04f8-4a03-91d1-860898570a3b"}]} {{(pid=125640) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 19 00:41:38.832702 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [req-159c42e2-780b-4f7e-a9c4-5ac1c5d7eddc req-1edeb3fc-540d-4d13-90be-cf99dae543bf service nova] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:41:38.833393 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [req-159c42e2-780b-4f7e-a9c4-5ac1c5d7eddc req-1edeb3fc-540d-4d13-90be-cf99dae543bf service nova] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:41:38.833999 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [req-159c42e2-780b-4f7e-a9c4-5ac1c5d7eddc req-1edeb3fc-540d-4d13-90be-cf99dae543bf service nova] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:41:38.880051 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: INFO nova.api.openstack.compute.server_external_events [req-159c42e2-780b-4f7e-a9c4-5ac1c5d7eddc req-1edeb3fc-540d-4d13-90be-cf99dae543bf service nova] Creating event network-vif-deleted:522599ca-04f8-4a03-91d1-860898570a3b for instance e8a28aad-697f-4dee-8254-6f08e9d76168 on hv-cinder-maste Oct 19 00:41:38.894034 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: INFO nova.api.openstack.requestlog [req-159c42e2-780b-4f7e-a9c4-5ac1c5d7eddc req-1edeb3fc-540d-4d13-90be-cf99dae543bf service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.118740 Oct 19 00:41:38.898236 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: [pid: 125640|app: 0|req: 68/137] 10.222.0.26 () {64 vars in 1317 bytes} [Wed Oct 19 00:41:38 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 125 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 19 00:41:39.524456 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-26536eea-8cf5-4969-8bce-57fc7560320c tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:41:39.525182 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-26536eea-8cf5-4969-8bce-57fc7560320c tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] [instance: e8a28aad-697f-4dee-8254-6f08e9d76168] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:41:39.566089 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-26536eea-8cf5-4969-8bce-57fc7560320c tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:41:39.566817 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-26536eea-8cf5-4969-8bce-57fc7560320c tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.026s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:41:39.567263 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-26536eea-8cf5-4969-8bce-57fc7560320c tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:41:39.571008 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-26536eea-8cf5-4969-8bce-57fc7560320c tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:41:39.576285 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-26536eea-8cf5-4969-8bce-57fc7560320c tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:41:39.581668 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-26536eea-8cf5-4969-8bce-57fc7560320c tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:41:39.665542 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi [None req-58b935c1-3ff1-40a0-9231-9e6968002221 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Calling method '>' {{(pid=125640) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:41:39.666733 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.compute.api [None req-58b935c1-3ff1-40a0-9231-9e6968002221 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] [instance: ab4c73d6-9203-4110-93af-4a78428c672b] Fetching instance by UUID {{(pid=125640) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:41:39.673972 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-26536eea-8cf5-4969-8bce-57fc7560320c tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:41:39.674707 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-26536eea-8cf5-4969-8bce-57fc7560320c tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:41:39.676999 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-26536eea-8cf5-4969-8bce-57fc7560320c tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:41:39.678701 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.objects.instance [None req-26536eea-8cf5-4969-8bce-57fc7560320c tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Lazy-loading 'fault' on Instance uuid e8a28aad-697f-4dee-8254-6f08e9d76168 {{(pid=125641) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 19 00:41:39.680944 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-58b935c1-3ff1-40a0-9231-9e6968002221 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:41:39.682411 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-58b935c1-3ff1-40a0-9231-9e6968002221 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:41:39.683117 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-58b935c1-3ff1-40a0-9231-9e6968002221 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:41:39.684535 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-58b935c1-3ff1-40a0-9231-9e6968002221 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:41:39.692147 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-58b935c1-3ff1-40a0-9231-9e6968002221 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:41:39.693757 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-58b935c1-3ff1-40a0-9231-9e6968002221 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.008s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:41:39.840777 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-26536eea-8cf5-4969-8bce-57fc7560320c tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8118f0c698aa428a93cb054abac30c2e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f249d9041c6749f58973a5b0867b07ce', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:41:39.860729 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-26536eea-8cf5-4969-8bce-57fc7560320c tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] 10.222.0.26 "GET /compute/v2.1/servers/e8a28aad-697f-4dee-8254-6f08e9d76168" status: 200 len: 1350 microversion: 2.1 time: 0.344771 Oct 19 00:41:39.861953 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 70/138] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:41:39 2022] GET /compute/v2.1/servers/e8a28aad-697f-4dee-8254-6f08e9d76168 => generated 1350 bytes in 349 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:41:39.978475 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.policy [None req-58b935c1-3ff1-40a0-9231-9e6968002221 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd3f0d615187d4779bdd7da8c9f24c796', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0151d58abd047a692f5cc4e7d05bea5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125640) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:41:39.998764 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: INFO nova.api.openstack.requestlog [None req-58b935c1-3ff1-40a0-9231-9e6968002221 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] 10.222.0.26 "GET /compute/v2.1/servers/ab4c73d6-9203-4110-93af-4a78428c672b" status: 200 len: 1522 microversion: 2.1 time: 0.339188 Oct 19 00:41:40.002681 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: [pid: 125640|app: 0|req: 69/139] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:41:39 2022] GET /compute/v2.1/servers/ab4c73d6-9203-4110-93af-4a78428c672b => generated 1522 bytes in 343 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 19 00:41:40.887683 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-74869458-5a4e-41a7-8b76-43fbf3dbf4b2 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:41:40.888884 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-74869458-5a4e-41a7-8b76-43fbf3dbf4b2 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] [instance: e8a28aad-697f-4dee-8254-6f08e9d76168] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:41:40.899605 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-74869458-5a4e-41a7-8b76-43fbf3dbf4b2 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:41:40.903020 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-74869458-5a4e-41a7-8b76-43fbf3dbf4b2 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:41:40.903906 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-74869458-5a4e-41a7-8b76-43fbf3dbf4b2 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:41:40.908506 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-74869458-5a4e-41a7-8b76-43fbf3dbf4b2 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:41:40.909168 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-74869458-5a4e-41a7-8b76-43fbf3dbf4b2 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:41:40.909614 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-74869458-5a4e-41a7-8b76-43fbf3dbf4b2 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:41:40.938709 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.wsgi [None req-74869458-5a4e-41a7-8b76-43fbf3dbf4b2 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] HTTP exception thrown: Instance e8a28aad-697f-4dee-8254-6f08e9d76168 could not be found. Oct 19 00:41:40.961993 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-74869458-5a4e-41a7-8b76-43fbf3dbf4b2 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Request method failure captured: Oct 19 00:41:40.961993 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: request: GET /compute/v2.1/servers/e8a28aad-697f-4dee-8254-6f08e9d76168 HTTP/1.1 Oct 19 00:41:40.961993 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Accept: application/json Oct 19 00:41:40.961993 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Accept-Encoding: identity Oct 19 00:41:40.961993 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Connection: close Oct 19 00:41:40.961993 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Content-Length: 0 Oct 19 00:41:40.961993 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Content-Type: application/json Oct 19 00:41:40.961993 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Host: 10.222.0.26 Oct 19 00:41:40.961993 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Openstack-System-Scope: None Oct 19 00:41:40.961993 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: User-Agent: python-urllib3/1.26.12 Oct 19 00:41:40.961993 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Auth-Token: gAAAAABjT0eK17qDuVF9Y5G3IdZqYwY-7C0pq0YOuZ-7cRSK1yG_DtvD44FcoN8Gv2ByRhfKhU1PsUk0kfduszsClrYdFAr_DoxGt54tO1JJ6Fb-rcKeJ_fPUoJT1wn_Q8kuPsz-2b0O8avFPAXv-K5U7M7F5T2pYgy9KP5BDwa40LGRRl3711U Oct 19 00:41:40.961993 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Domain-Id: None Oct 19 00:41:40.961993 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Domain-Name: None Oct 19 00:41:40.961993 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Identity-Status: Confirmed Oct 19 00:41:40.961993 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Is-Admin-Project: True Oct 19 00:41:40.961993 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Project-Domain-Id: default Oct 19 00:41:40.961993 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Project-Domain-Name: Default Oct 19 00:41:40.961993 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Project-Id: f249d9041c6749f58973a5b0867b07ce Oct 19 00:41:40.961993 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Project-Name: tempest-DeleteServersTestJSON-1183919440 Oct 19 00:41:40.961993 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Role: member,reader Oct 19 00:41:40.961993 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Roles: member,reader Oct 19 00:41:40.963630 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/f249d9041c6749f58973a5b0867b07ce"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/f249d9041c6749f58973a5b0867b07ce"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/f249d9041c6749f58973a5b0867b07ce"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_f249d9041c6749f58973a5b0867b07ce"}]}] Oct 19 00:41:40.963630 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Tenant: tempest-DeleteServersTestJSON-1183919440 Oct 19 00:41:40.963630 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Tenant-Id: f249d9041c6749f58973a5b0867b07ce Oct 19 00:41:40.963630 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Tenant-Name: tempest-DeleteServersTestJSON-1183919440 Oct 19 00:41:40.963630 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User: tempest-DeleteServersTestJSON-1183919440-project Oct 19 00:41:40.963630 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User-Domain-Id: default Oct 19 00:41:40.963630 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User-Domain-Name: Default Oct 19 00:41:40.963630 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User-Id: 8118f0c698aa428a93cb054abac30c2e Oct 19 00:41:40.963630 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User-Name: tempest-DeleteServersTestJSON-1183919440-project Oct 19 00:41:40.963630 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: method: > Oct 19 00:41:40.963630 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: exception: Instance e8a28aad-697f-4dee-8254-6f08e9d76168 could not be found. Oct 19 00:41:40.963630 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: action_args: {'id': 'e8a28aad-697f-4dee-8254-6f08e9d76168'} Oct 19 00:41:40.964488 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 19 00:41:40.964488 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 19 00:41:40.964488 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 19 00:41:40.964488 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 19 00:41:40.964488 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 19 00:41:40.964488 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 19 00:41:40.964488 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 19 00:41:40.964488 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 19 00:41:40.964488 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 19 00:41:40.964488 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 19 00:41:40.964488 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance e8a28aad-697f-4dee-8254-6f08e9d76168 could not be found. Oct 19 00:41:40.964488 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:41:40.964488 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 19 00:41:40.964488 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:41:40.964488 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 19 00:41:40.964488 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 19 00:41:40.964488 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 19 00:41:40.964488 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 19 00:41:40.964488 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 19 00:41:40.964488 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 19 00:41:40.966159 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 19 00:41:40.966159 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 19 00:41:40.966159 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 19 00:41:40.966159 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 19 00:41:40.966159 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 19 00:41:40.966159 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 19 00:41:40.966159 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 19 00:41:40.966159 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance e8a28aad-697f-4dee-8254-6f08e9d76168 could not be found. Oct 19 00:41:40.966159 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:41:40.966159 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 19 00:41:40.966159 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:41:40.966159 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 19 00:41:40.966159 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 19 00:41:40.966159 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 19 00:41:40.966159 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 19 00:41:40.966159 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 19 00:41:40.966159 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance e8a28aad-697f-4dee-8254-6f08e9d76168 could not be found. Oct 19 00:41:40.966159 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:41:40.967394 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-74869458-5a4e-41a7-8b76-43fbf3dbf4b2 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Returning 404 to user: Instance e8a28aad-697f-4dee-8254-6f08e9d76168 could not be found. {{(pid=125641) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 19 00:41:40.968574 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-74869458-5a4e-41a7-8b76-43fbf3dbf4b2 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] 10.222.0.26 "GET /compute/v2.1/servers/e8a28aad-697f-4dee-8254-6f08e9d76168" status: 404 len: 111 microversion: 2.1 time: 0.085620 Oct 19 00:41:40.969935 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 71/140] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:41:40 2022] GET /compute/v2.1/servers/e8a28aad-697f-4dee-8254-6f08e9d76168 => generated 111 bytes in 87 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 19 00:41:41.021860 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi [None req-334b8ec0-ce37-4c89-932c-06c72984f612 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Calling method '>' {{(pid=125640) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:41:41.022471 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.compute.api [None req-334b8ec0-ce37-4c89-932c-06c72984f612 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] [instance: ab4c73d6-9203-4110-93af-4a78428c672b] Fetching instance by UUID {{(pid=125640) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:41:41.031445 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-334b8ec0-ce37-4c89-932c-06c72984f612 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:41:41.031985 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-334b8ec0-ce37-4c89-932c-06c72984f612 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:41:41.033411 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-334b8ec0-ce37-4c89-932c-06c72984f612 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:41:41.034563 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-334b8ec0-ce37-4c89-932c-06c72984f612 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:41:41.036270 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-334b8ec0-ce37-4c89-932c-06c72984f612 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:41:41.041971 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-334b8ec0-ce37-4c89-932c-06c72984f612 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:41:41.167344 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-fa12b3a9-8588-4208-8964-f2bed3ed9406 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:41:41.170347 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-fa12b3a9-8588-4208-8964-f2bed3ed9406 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] [instance: e8a28aad-697f-4dee-8254-6f08e9d76168] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:41:41.202179 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-fa12b3a9-8588-4208-8964-f2bed3ed9406 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:41:41.209697 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-fa12b3a9-8588-4208-8964-f2bed3ed9406 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:41:41.211730 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-fa12b3a9-8588-4208-8964-f2bed3ed9406 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:41:41.216158 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-fa12b3a9-8588-4208-8964-f2bed3ed9406 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:41:41.217276 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-fa12b3a9-8588-4208-8964-f2bed3ed9406 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:41:41.217906 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-fa12b3a9-8588-4208-8964-f2bed3ed9406 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:41:41.276833 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.wsgi [None req-fa12b3a9-8588-4208-8964-f2bed3ed9406 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] HTTP exception thrown: Instance e8a28aad-697f-4dee-8254-6f08e9d76168 could not be found. Oct 19 00:41:41.285247 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-fa12b3a9-8588-4208-8964-f2bed3ed9406 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Request method failure captured: Oct 19 00:41:41.285247 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: request: GET /compute/v2.1/servers/e8a28aad-697f-4dee-8254-6f08e9d76168/os-volume_attachments HTTP/1.1 Oct 19 00:41:41.285247 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Accept: application/json Oct 19 00:41:41.285247 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Accept-Encoding: identity Oct 19 00:41:41.285247 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Connection: close Oct 19 00:41:41.285247 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Content-Length: 0 Oct 19 00:41:41.285247 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Content-Type: application/json Oct 19 00:41:41.285247 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Host: 10.222.0.26 Oct 19 00:41:41.285247 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Openstack-System-Scope: None Oct 19 00:41:41.285247 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: User-Agent: python-urllib3/1.26.12 Oct 19 00:41:41.285247 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Auth-Token: gAAAAABjT0eK17qDuVF9Y5G3IdZqYwY-7C0pq0YOuZ-7cRSK1yG_DtvD44FcoN8Gv2ByRhfKhU1PsUk0kfduszsClrYdFAr_DoxGt54tO1JJ6Fb-rcKeJ_fPUoJT1wn_Q8kuPsz-2b0O8avFPAXv-K5U7M7F5T2pYgy9KP5BDwa40LGRRl3711U Oct 19 00:41:41.285247 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Domain-Id: None Oct 19 00:41:41.285247 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Domain-Name: None Oct 19 00:41:41.285247 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Identity-Status: Confirmed Oct 19 00:41:41.285247 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Is-Admin-Project: True Oct 19 00:41:41.285247 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Project-Domain-Id: default Oct 19 00:41:41.285247 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Project-Domain-Name: Default Oct 19 00:41:41.285247 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Project-Id: f249d9041c6749f58973a5b0867b07ce Oct 19 00:41:41.285247 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Project-Name: tempest-DeleteServersTestJSON-1183919440 Oct 19 00:41:41.285247 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Role: member,reader Oct 19 00:41:41.285247 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Roles: member,reader Oct 19 00:41:41.286727 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/f249d9041c6749f58973a5b0867b07ce"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/f249d9041c6749f58973a5b0867b07ce"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/f249d9041c6749f58973a5b0867b07ce"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_f249d9041c6749f58973a5b0867b07ce"}]}] Oct 19 00:41:41.286727 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Tenant: tempest-DeleteServersTestJSON-1183919440 Oct 19 00:41:41.286727 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Tenant-Id: f249d9041c6749f58973a5b0867b07ce Oct 19 00:41:41.286727 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Tenant-Name: tempest-DeleteServersTestJSON-1183919440 Oct 19 00:41:41.286727 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User: tempest-DeleteServersTestJSON-1183919440-project Oct 19 00:41:41.286727 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User-Domain-Id: default Oct 19 00:41:41.286727 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User-Domain-Name: Default Oct 19 00:41:41.286727 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User-Id: 8118f0c698aa428a93cb054abac30c2e Oct 19 00:41:41.286727 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User-Name: tempest-DeleteServersTestJSON-1183919440-project Oct 19 00:41:41.286727 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: method: > Oct 19 00:41:41.286727 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: exception: Instance e8a28aad-697f-4dee-8254-6f08e9d76168 could not be found. Oct 19 00:41:41.286727 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: action_args: {'server_id': 'e8a28aad-697f-4dee-8254-6f08e9d76168'} Oct 19 00:41:41.287533 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 19 00:41:41.287533 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 19 00:41:41.287533 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 19 00:41:41.287533 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 19 00:41:41.287533 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 19 00:41:41.287533 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 19 00:41:41.287533 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 19 00:41:41.287533 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 19 00:41:41.287533 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 19 00:41:41.287533 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 19 00:41:41.287533 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance e8a28aad-697f-4dee-8254-6f08e9d76168 could not be found. Oct 19 00:41:41.287533 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:41:41.287533 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 19 00:41:41.287533 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:41:41.287533 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 19 00:41:41.287533 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 19 00:41:41.287533 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 19 00:41:41.287533 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 19 00:41:41.287533 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 19 00:41:41.287533 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 19 00:41:41.288900 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 19 00:41:41.288900 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 192, in wrapper Oct 19 00:41:41.288900 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 19 00:41:41.288900 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 192, in wrapper Oct 19 00:41:41.288900 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 19 00:41:41.288900 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 303, in index Oct 19 00:41:41.288900 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, server_id) Oct 19 00:41:41.288900 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 19 00:41:41.288900 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 19 00:41:41.288900 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance e8a28aad-697f-4dee-8254-6f08e9d76168 could not be found. Oct 19 00:41:41.288900 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:41:41.288900 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 19 00:41:41.288900 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:41:41.288900 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 19 00:41:41.288900 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 19 00:41:41.288900 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 19 00:41:41.288900 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 19 00:41:41.288900 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 19 00:41:41.288900 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance e8a28aad-697f-4dee-8254-6f08e9d76168 could not be found. Oct 19 00:41:41.288900 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:41:41.290664 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-fa12b3a9-8588-4208-8964-f2bed3ed9406 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Returning 404 to user: Instance e8a28aad-697f-4dee-8254-6f08e9d76168 could not be found. {{(pid=125641) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 19 00:41:41.291787 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-fa12b3a9-8588-4208-8964-f2bed3ed9406 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] 10.222.0.26 "GET /compute/v2.1/servers/e8a28aad-697f-4dee-8254-6f08e9d76168/os-volume_attachments" status: 404 len: 111 microversion: 2.1 time: 0.129418 Oct 19 00:41:41.297724 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 72/141] 10.222.0.26 () {58 vars in 1302 bytes} [Wed Oct 19 00:41:41 2022] GET /compute/v2.1/servers/e8a28aad-697f-4dee-8254-6f08e9d76168/os-volume_attachments => generated 111 bytes in 136 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 19 00:41:41.454012 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.policy [None req-334b8ec0-ce37-4c89-932c-06c72984f612 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd3f0d615187d4779bdd7da8c9f24c796', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0151d58abd047a692f5cc4e7d05bea5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125640) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:41:41.468433 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: INFO nova.api.openstack.requestlog [None req-334b8ec0-ce37-4c89-932c-06c72984f612 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] 10.222.0.26 "GET /compute/v2.1/servers/ab4c73d6-9203-4110-93af-4a78428c672b" status: 200 len: 1522 microversion: 2.1 time: 0.450324 Oct 19 00:41:41.469076 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: [pid: 125640|app: 0|req: 70/142] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:41:41 2022] GET /compute/v2.1/servers/ab4c73d6-9203-4110-93af-4a78428c672b => generated 1522 bytes in 452 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:41:42.497631 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-da3a0a5c-2815-4fb0-83e8-3df7aab6fedf tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:41:42.498464 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-da3a0a5c-2815-4fb0-83e8-3df7aab6fedf tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] [instance: ab4c73d6-9203-4110-93af-4a78428c672b] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:41:42.509655 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-da3a0a5c-2815-4fb0-83e8-3df7aab6fedf tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:41:42.510872 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-da3a0a5c-2815-4fb0-83e8-3df7aab6fedf tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:41:42.511280 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-da3a0a5c-2815-4fb0-83e8-3df7aab6fedf tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:41:42.512060 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-da3a0a5c-2815-4fb0-83e8-3df7aab6fedf tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:41:42.518015 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-da3a0a5c-2815-4fb0-83e8-3df7aab6fedf tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:41:42.519279 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-da3a0a5c-2815-4fb0-83e8-3df7aab6fedf tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:41:42.784267 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-da3a0a5c-2815-4fb0-83e8-3df7aab6fedf tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd3f0d615187d4779bdd7da8c9f24c796', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0151d58abd047a692f5cc4e7d05bea5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:41:42.796032 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-da3a0a5c-2815-4fb0-83e8-3df7aab6fedf tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] 10.222.0.26 "GET /compute/v2.1/servers/ab4c73d6-9203-4110-93af-4a78428c672b" status: 200 len: 1522 microversion: 2.1 time: 0.303232 Oct 19 00:41:42.798525 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 73/143] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:41:42 2022] GET /compute/v2.1/servers/ab4c73d6-9203-4110-93af-4a78428c672b => generated 1522 bytes in 306 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:41:43.833032 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi [None req-252b652a-49b0-4cfd-a7d5-03c32c2726b0 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Calling method '>' {{(pid=125640) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:41:43.833843 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.compute.api [None req-252b652a-49b0-4cfd-a7d5-03c32c2726b0 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] [instance: ab4c73d6-9203-4110-93af-4a78428c672b] Fetching instance by UUID {{(pid=125640) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:41:43.842168 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-252b652a-49b0-4cfd-a7d5-03c32c2726b0 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:41:43.843070 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-252b652a-49b0-4cfd-a7d5-03c32c2726b0 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:41:43.844107 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-252b652a-49b0-4cfd-a7d5-03c32c2726b0 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:41:43.845782 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-252b652a-49b0-4cfd-a7d5-03c32c2726b0 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:41:43.847136 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-252b652a-49b0-4cfd-a7d5-03c32c2726b0 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:41:43.847805 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-252b652a-49b0-4cfd-a7d5-03c32c2726b0 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:41:44.136553 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.policy [None req-252b652a-49b0-4cfd-a7d5-03c32c2726b0 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd3f0d615187d4779bdd7da8c9f24c796', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0151d58abd047a692f5cc4e7d05bea5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125640) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:41:44.146438 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: INFO nova.api.openstack.requestlog [None req-252b652a-49b0-4cfd-a7d5-03c32c2726b0 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] 10.222.0.26 "GET /compute/v2.1/servers/ab4c73d6-9203-4110-93af-4a78428c672b" status: 200 len: 1522 microversion: 2.1 time: 0.327088 Oct 19 00:41:44.147096 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: [pid: 125640|app: 0|req: 71/144] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:41:43 2022] GET /compute/v2.1/servers/ab4c73d6-9203-4110-93af-4a78428c672b => generated 1522 bytes in 329 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:41:44.924536 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-177eb5fd-c651-4bd3-b84e-a0f58ce5f327 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:41:44.925665 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-177eb5fd-c651-4bd3-b84e-a0f58ce5f327 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] [instance: e8a28aad-697f-4dee-8254-6f08e9d76168] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:41:44.940870 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-177eb5fd-c651-4bd3-b84e-a0f58ce5f327 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:41:44.941805 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-177eb5fd-c651-4bd3-b84e-a0f58ce5f327 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:41:44.942628 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-177eb5fd-c651-4bd3-b84e-a0f58ce5f327 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:41:44.943909 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-177eb5fd-c651-4bd3-b84e-a0f58ce5f327 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:41:44.944697 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-177eb5fd-c651-4bd3-b84e-a0f58ce5f327 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:41:44.945858 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-177eb5fd-c651-4bd3-b84e-a0f58ce5f327 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:41:45.006744 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.wsgi [None req-177eb5fd-c651-4bd3-b84e-a0f58ce5f327 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] HTTP exception thrown: Instance e8a28aad-697f-4dee-8254-6f08e9d76168 could not be found. Oct 19 00:41:45.010068 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-177eb5fd-c651-4bd3-b84e-a0f58ce5f327 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Request method failure captured: Oct 19 00:41:45.010068 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: request: DELETE /compute/v2.1/servers/e8a28aad-697f-4dee-8254-6f08e9d76168 HTTP/1.1 Oct 19 00:41:45.010068 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Accept: application/json Oct 19 00:41:45.010068 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Accept-Encoding: identity Oct 19 00:41:45.010068 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Connection: close Oct 19 00:41:45.010068 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Content-Length: 0 Oct 19 00:41:45.010068 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Content-Type: application/json Oct 19 00:41:45.010068 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Host: 10.222.0.26 Oct 19 00:41:45.010068 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Openstack-System-Scope: None Oct 19 00:41:45.010068 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: User-Agent: python-urllib3/1.26.12 Oct 19 00:41:45.010068 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Auth-Token: gAAAAABjT0eK17qDuVF9Y5G3IdZqYwY-7C0pq0YOuZ-7cRSK1yG_DtvD44FcoN8Gv2ByRhfKhU1PsUk0kfduszsClrYdFAr_DoxGt54tO1JJ6Fb-rcKeJ_fPUoJT1wn_Q8kuPsz-2b0O8avFPAXv-K5U7M7F5T2pYgy9KP5BDwa40LGRRl3711U Oct 19 00:41:45.010068 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Domain-Id: None Oct 19 00:41:45.010068 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Domain-Name: None Oct 19 00:41:45.010068 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Identity-Status: Confirmed Oct 19 00:41:45.010068 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Is-Admin-Project: True Oct 19 00:41:45.010068 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Project-Domain-Id: default Oct 19 00:41:45.010068 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Project-Domain-Name: Default Oct 19 00:41:45.010068 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Project-Id: f249d9041c6749f58973a5b0867b07ce Oct 19 00:41:45.010068 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Project-Name: tempest-DeleteServersTestJSON-1183919440 Oct 19 00:41:45.010068 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Role: member,reader Oct 19 00:41:45.010068 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Roles: member,reader Oct 19 00:41:45.012841 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/f249d9041c6749f58973a5b0867b07ce"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/f249d9041c6749f58973a5b0867b07ce"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/f249d9041c6749f58973a5b0867b07ce"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_f249d9041c6749f58973a5b0867b07ce"}]}] Oct 19 00:41:45.012841 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Tenant: tempest-DeleteServersTestJSON-1183919440 Oct 19 00:41:45.012841 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Tenant-Id: f249d9041c6749f58973a5b0867b07ce Oct 19 00:41:45.012841 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Tenant-Name: tempest-DeleteServersTestJSON-1183919440 Oct 19 00:41:45.012841 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User: tempest-DeleteServersTestJSON-1183919440-project Oct 19 00:41:45.012841 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User-Domain-Id: default Oct 19 00:41:45.012841 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User-Domain-Name: Default Oct 19 00:41:45.012841 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User-Id: 8118f0c698aa428a93cb054abac30c2e Oct 19 00:41:45.012841 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User-Name: tempest-DeleteServersTestJSON-1183919440-project Oct 19 00:41:45.012841 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: method: > Oct 19 00:41:45.012841 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: exception: Instance e8a28aad-697f-4dee-8254-6f08e9d76168 could not be found. Oct 19 00:41:45.012841 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: action_args: {'id': 'e8a28aad-697f-4dee-8254-6f08e9d76168'} Oct 19 00:41:45.014380 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 19 00:41:45.014380 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 19 00:41:45.014380 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 19 00:41:45.014380 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 19 00:41:45.014380 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 19 00:41:45.014380 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 19 00:41:45.014380 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 19 00:41:45.014380 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 19 00:41:45.014380 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 19 00:41:45.014380 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 19 00:41:45.014380 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance e8a28aad-697f-4dee-8254-6f08e9d76168 could not be found. Oct 19 00:41:45.014380 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:41:45.014380 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 19 00:41:45.014380 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:41:45.014380 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 19 00:41:45.014380 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 19 00:41:45.014380 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 19 00:41:45.014380 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 19 00:41:45.014380 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 19 00:41:45.014380 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 19 00:41:45.018690 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 19 00:41:45.018690 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Oct 19 00:41:45.018690 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Oct 19 00:41:45.018690 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Oct 19 00:41:45.018690 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Oct 19 00:41:45.018690 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 19 00:41:45.018690 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 19 00:41:45.018690 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 19 00:41:45.018690 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 19 00:41:45.018690 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance e8a28aad-697f-4dee-8254-6f08e9d76168 could not be found. Oct 19 00:41:45.018690 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:41:45.018690 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 19 00:41:45.018690 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:41:45.018690 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 19 00:41:45.018690 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 19 00:41:45.018690 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 19 00:41:45.018690 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 19 00:41:45.018690 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 19 00:41:45.018690 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance e8a28aad-697f-4dee-8254-6f08e9d76168 could not be found. Oct 19 00:41:45.018690 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:41:45.026143 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-177eb5fd-c651-4bd3-b84e-a0f58ce5f327 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Returning 404 to user: Instance e8a28aad-697f-4dee-8254-6f08e9d76168 could not be found. {{(pid=125641) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 19 00:41:45.026143 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-177eb5fd-c651-4bd3-b84e-a0f58ce5f327 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] 10.222.0.26 "DELETE /compute/v2.1/servers/e8a28aad-697f-4dee-8254-6f08e9d76168" status: 404 len: 111 microversion: 2.1 time: 0.114089 Oct 19 00:41:45.027761 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 74/145] 10.222.0.26 () {58 vars in 1239 bytes} [Wed Oct 19 00:41:44 2022] DELETE /compute/v2.1/servers/e8a28aad-697f-4dee-8254-6f08e9d76168 => generated 111 bytes in 121 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 19 00:41:45.039137 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi [None req-6896f1a6-132e-4895-b7f3-55f5658e5c43 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Calling method '>' {{(pid=125640) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:41:45.043369 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.compute.api [None req-6896f1a6-132e-4895-b7f3-55f5658e5c43 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] [instance: e8a28aad-697f-4dee-8254-6f08e9d76168] Fetching instance by UUID {{(pid=125640) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:41:45.085733 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-6896f1a6-132e-4895-b7f3-55f5658e5c43 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:41:45.087362 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-6896f1a6-132e-4895-b7f3-55f5658e5c43 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.012s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:41:45.088578 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-6896f1a6-132e-4895-b7f3-55f5658e5c43 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:41:45.090615 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-6896f1a6-132e-4895-b7f3-55f5658e5c43 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:41:45.091663 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-6896f1a6-132e-4895-b7f3-55f5658e5c43 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:41:45.092395 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-6896f1a6-132e-4895-b7f3-55f5658e5c43 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:41:45.138535 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: INFO nova.api.openstack.wsgi [None req-6896f1a6-132e-4895-b7f3-55f5658e5c43 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] HTTP exception thrown: Instance e8a28aad-697f-4dee-8254-6f08e9d76168 could not be found. Oct 19 00:41:45.147426 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi [None req-6896f1a6-132e-4895-b7f3-55f5658e5c43 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Request method failure captured: Oct 19 00:41:45.147426 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: request: GET /compute/v2.1/servers/e8a28aad-697f-4dee-8254-6f08e9d76168 HTTP/1.1 Oct 19 00:41:45.147426 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: Accept: application/json Oct 19 00:41:45.147426 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: Accept-Encoding: identity Oct 19 00:41:45.147426 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: Connection: close Oct 19 00:41:45.147426 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: Content-Length: 0 Oct 19 00:41:45.147426 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: Content-Type: application/json Oct 19 00:41:45.147426 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: Host: 10.222.0.26 Oct 19 00:41:45.147426 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: Openstack-System-Scope: None Oct 19 00:41:45.147426 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: User-Agent: python-urllib3/1.26.12 Oct 19 00:41:45.147426 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: X-Auth-Token: gAAAAABjT0eK17qDuVF9Y5G3IdZqYwY-7C0pq0YOuZ-7cRSK1yG_DtvD44FcoN8Gv2ByRhfKhU1PsUk0kfduszsClrYdFAr_DoxGt54tO1JJ6Fb-rcKeJ_fPUoJT1wn_Q8kuPsz-2b0O8avFPAXv-K5U7M7F5T2pYgy9KP5BDwa40LGRRl3711U Oct 19 00:41:45.147426 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: X-Domain-Id: None Oct 19 00:41:45.147426 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: X-Domain-Name: None Oct 19 00:41:45.147426 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: X-Identity-Status: Confirmed Oct 19 00:41:45.147426 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: X-Is-Admin-Project: True Oct 19 00:41:45.147426 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: X-Project-Domain-Id: default Oct 19 00:41:45.147426 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: X-Project-Domain-Name: Default Oct 19 00:41:45.147426 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: X-Project-Id: f249d9041c6749f58973a5b0867b07ce Oct 19 00:41:45.147426 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: X-Project-Name: tempest-DeleteServersTestJSON-1183919440 Oct 19 00:41:45.147426 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: X-Role: member,reader Oct 19 00:41:45.147426 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: X-Roles: member,reader Oct 19 00:41:45.149575 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/f249d9041c6749f58973a5b0867b07ce"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/f249d9041c6749f58973a5b0867b07ce"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/f249d9041c6749f58973a5b0867b07ce"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_f249d9041c6749f58973a5b0867b07ce"}]}] Oct 19 00:41:45.149575 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: X-Tenant: tempest-DeleteServersTestJSON-1183919440 Oct 19 00:41:45.149575 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: X-Tenant-Id: f249d9041c6749f58973a5b0867b07ce Oct 19 00:41:45.149575 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: X-Tenant-Name: tempest-DeleteServersTestJSON-1183919440 Oct 19 00:41:45.149575 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: X-User: tempest-DeleteServersTestJSON-1183919440-project Oct 19 00:41:45.149575 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: X-User-Domain-Id: default Oct 19 00:41:45.149575 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: X-User-Domain-Name: Default Oct 19 00:41:45.149575 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: X-User-Id: 8118f0c698aa428a93cb054abac30c2e Oct 19 00:41:45.149575 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: X-User-Name: tempest-DeleteServersTestJSON-1183919440-project Oct 19 00:41:45.149575 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: method: > Oct 19 00:41:45.149575 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: exception: Instance e8a28aad-697f-4dee-8254-6f08e9d76168 could not be found. Oct 19 00:41:45.149575 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: action_args: {'id': 'e8a28aad-697f-4dee-8254-6f08e9d76168'} Oct 19 00:41:45.151066 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: {{(pid=125640) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 19 00:41:45.151066 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 19 00:41:45.151066 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 19 00:41:45.151066 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 19 00:41:45.151066 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 19 00:41:45.151066 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 19 00:41:45.151066 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 19 00:41:45.151066 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 19 00:41:45.151066 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 19 00:41:45.151066 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 19 00:41:45.151066 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance e8a28aad-697f-4dee-8254-6f08e9d76168 could not be found. Oct 19 00:41:45.151066 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi Oct 19 00:41:45.151066 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 19 00:41:45.151066 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi Oct 19 00:41:45.151066 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 19 00:41:45.151066 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 19 00:41:45.151066 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 19 00:41:45.151066 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 19 00:41:45.151066 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 19 00:41:45.151066 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 19 00:41:45.155258 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 19 00:41:45.155258 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 19 00:41:45.155258 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 19 00:41:45.155258 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 19 00:41:45.155258 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 19 00:41:45.155258 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 19 00:41:45.155258 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 19 00:41:45.155258 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance e8a28aad-697f-4dee-8254-6f08e9d76168 could not be found. Oct 19 00:41:45.155258 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi Oct 19 00:41:45.155258 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 19 00:41:45.155258 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi Oct 19 00:41:45.155258 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 19 00:41:45.155258 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 19 00:41:45.155258 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 19 00:41:45.155258 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 19 00:41:45.155258 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 19 00:41:45.155258 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance e8a28aad-697f-4dee-8254-6f08e9d76168 could not be found. Oct 19 00:41:45.155258 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi Oct 19 00:41:45.176199 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi [None req-6896f1a6-132e-4895-b7f3-55f5658e5c43 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] Returning 404 to user: Instance e8a28aad-697f-4dee-8254-6f08e9d76168 could not be found. {{(pid=125640) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 19 00:41:45.180328 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-f3ba0f3a-d6d1-436c-814d-858c7ac98281 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:41:45.182608 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: INFO nova.api.openstack.requestlog [None req-6896f1a6-132e-4895-b7f3-55f5658e5c43 tempest-DeleteServersTestJSON-1183919440 tempest-DeleteServersTestJSON-1183919440-project] 10.222.0.26 "GET /compute/v2.1/servers/e8a28aad-697f-4dee-8254-6f08e9d76168" status: 404 len: 111 microversion: 2.1 time: 0.145142 Oct 19 00:41:45.183517 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: [pid: 125640|app: 0|req: 72/146] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:41:45 2022] GET /compute/v2.1/servers/e8a28aad-697f-4dee-8254-6f08e9d76168 => generated 111 bytes in 152 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 19 00:41:45.184187 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-f3ba0f3a-d6d1-436c-814d-858c7ac98281 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] [instance: ab4c73d6-9203-4110-93af-4a78428c672b] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:41:45.230169 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-f3ba0f3a-d6d1-436c-814d-858c7ac98281 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:41:45.231847 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-f3ba0f3a-d6d1-436c-814d-858c7ac98281 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:41:45.232912 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-f3ba0f3a-d6d1-436c-814d-858c7ac98281 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:41:45.236134 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-f3ba0f3a-d6d1-436c-814d-858c7ac98281 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:41:45.237946 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-f3ba0f3a-d6d1-436c-814d-858c7ac98281 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:41:45.239485 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-f3ba0f3a-d6d1-436c-814d-858c7ac98281 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:41:46.053874 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-f3ba0f3a-d6d1-436c-814d-858c7ac98281 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd3f0d615187d4779bdd7da8c9f24c796', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0151d58abd047a692f5cc4e7d05bea5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:41:46.064896 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-f3ba0f3a-d6d1-436c-814d-858c7ac98281 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] 10.222.0.26 "GET /compute/v2.1/servers/ab4c73d6-9203-4110-93af-4a78428c672b" status: 200 len: 1522 microversion: 2.1 time: 0.895140 Oct 19 00:41:46.083595 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 75/147] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:41:45 2022] GET /compute/v2.1/servers/ab4c73d6-9203-4110-93af-4a78428c672b => generated 1522 bytes in 916 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:41:47.093246 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi [None req-2a94f186-e80f-44b8-8ccb-bc3ac427e334 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Calling method '>' {{(pid=125640) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:41:47.094792 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.compute.api [None req-2a94f186-e80f-44b8-8ccb-bc3ac427e334 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] [instance: ab4c73d6-9203-4110-93af-4a78428c672b] Fetching instance by UUID {{(pid=125640) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:41:47.104795 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-2a94f186-e80f-44b8-8ccb-bc3ac427e334 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:41:47.112399 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-2a94f186-e80f-44b8-8ccb-bc3ac427e334 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:41:47.113021 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-2a94f186-e80f-44b8-8ccb-bc3ac427e334 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:41:47.114540 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-2a94f186-e80f-44b8-8ccb-bc3ac427e334 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:41:47.115239 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-2a94f186-e80f-44b8-8ccb-bc3ac427e334 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:41:47.115976 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-2a94f186-e80f-44b8-8ccb-bc3ac427e334 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:41:47.439523 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.policy [None req-2a94f186-e80f-44b8-8ccb-bc3ac427e334 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd3f0d615187d4779bdd7da8c9f24c796', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0151d58abd047a692f5cc4e7d05bea5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125640) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:41:47.463794 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: INFO nova.api.openstack.requestlog [None req-2a94f186-e80f-44b8-8ccb-bc3ac427e334 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] 10.222.0.26 "GET /compute/v2.1/servers/ab4c73d6-9203-4110-93af-4a78428c672b" status: 200 len: 1522 microversion: 2.1 time: 0.367296 Oct 19 00:41:47.472700 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: [pid: 125640|app: 0|req: 73/148] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:41:47 2022] GET /compute/v2.1/servers/ab4c73d6-9203-4110-93af-4a78428c672b => generated 1522 bytes in 384 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:41:48.495560 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-992a9882-e9cd-4e10-8bda-f5d38a948fe5 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:41:48.498590 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-992a9882-e9cd-4e10-8bda-f5d38a948fe5 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] [instance: ab4c73d6-9203-4110-93af-4a78428c672b] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:41:48.506701 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-992a9882-e9cd-4e10-8bda-f5d38a948fe5 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:41:48.507409 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-992a9882-e9cd-4e10-8bda-f5d38a948fe5 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:41:48.507903 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-992a9882-e9cd-4e10-8bda-f5d38a948fe5 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:41:48.508673 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-992a9882-e9cd-4e10-8bda-f5d38a948fe5 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:41:48.509421 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-992a9882-e9cd-4e10-8bda-f5d38a948fe5 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:41:48.509906 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-992a9882-e9cd-4e10-8bda-f5d38a948fe5 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:41:48.834537 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-992a9882-e9cd-4e10-8bda-f5d38a948fe5 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd3f0d615187d4779bdd7da8c9f24c796', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0151d58abd047a692f5cc4e7d05bea5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:41:48.847380 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-992a9882-e9cd-4e10-8bda-f5d38a948fe5 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] 10.222.0.26 "GET /compute/v2.1/servers/ab4c73d6-9203-4110-93af-4a78428c672b" status: 200 len: 1522 microversion: 2.1 time: 0.355893 Oct 19 00:41:48.852020 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 76/149] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:41:48 2022] GET /compute/v2.1/servers/ab4c73d6-9203-4110-93af-4a78428c672b => generated 1522 bytes in 361 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:41:49.080815 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi [None req-b53c1e04-f1ec-4e49-9a60-3b06aac6f6bc tempest-TestShelveInstance-770582455 tempest-TestShelveInstance-770582455-project] Action: 'create', calling method: .version_select at 0x7f71c3f5ad30>, body: {"keypair": {"name": "tempest-TestShelveInstance-480977406", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBMEvoguP84vFLjXskkGBZA7FJIt9YiZ0XMzINsRRaqDkKfBXWBnqPOL2a4iOnhA3qqtV43VwZ2MSs16IVpDA35pT+74ywPgp7DptdXs7ZGsERX3883olguOKobqxpruLCA=="}} {{(pid=125640) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 19 00:41:49.127043 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.quota [None req-b53c1e04-f1ec-4e49-9a60-3b06aac6f6bc tempest-TestShelveInstance-770582455 tempest-TestShelveInstance-770582455-project] Getting quotas for project f798b16ae6e4429c9c07793a0178f798. Resources: {'key_pairs'} {{(pid=125640) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 19 00:41:49.155596 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.quota [None req-b53c1e04-f1ec-4e49-9a60-3b06aac6f6bc tempest-TestShelveInstance-770582455 tempest-TestShelveInstance-770582455-project] Getting quotas for user ed3d6b4184274791bce9c7d9b1ab028a and project f798b16ae6e4429c9c07793a0178f798. Resources: {'key_pairs'} {{(pid=125640) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 19 00:41:49.194732 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: INFO nova.api.openstack.requestlog [None req-b53c1e04-f1ec-4e49-9a60-3b06aac6f6bc tempest-TestShelveInstance-770582455 tempest-TestShelveInstance-770582455-project] 10.222.0.26 "POST /compute/v2.1/os-keypairs" status: 200 len: 396 microversion: 2.1 time: 0.453699 Oct 19 00:41:49.195709 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: [pid: 125640|app: 0|req: 74/150] 10.222.0.26 () {60 vars in 1159 bytes} [Wed Oct 19 00:41:48 2022] POST /compute/v2.1/os-keypairs => generated 396 bytes in 455 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 19 00:41:49.876032 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-768196f2-52d8-45dc-acf8-2c9a13840e95 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:41:49.877477 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-768196f2-52d8-45dc-acf8-2c9a13840e95 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] [instance: ab4c73d6-9203-4110-93af-4a78428c672b] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:41:49.888791 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-768196f2-52d8-45dc-acf8-2c9a13840e95 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:41:49.891077 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-768196f2-52d8-45dc-acf8-2c9a13840e95 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:41:49.893365 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-768196f2-52d8-45dc-acf8-2c9a13840e95 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:41:49.894368 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-768196f2-52d8-45dc-acf8-2c9a13840e95 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:41:49.896314 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-768196f2-52d8-45dc-acf8-2c9a13840e95 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:41:49.896992 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-768196f2-52d8-45dc-acf8-2c9a13840e95 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:41:50.252263 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-768196f2-52d8-45dc-acf8-2c9a13840e95 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd3f0d615187d4779bdd7da8c9f24c796', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0151d58abd047a692f5cc4e7d05bea5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:41:50.266621 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-768196f2-52d8-45dc-acf8-2c9a13840e95 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] 10.222.0.26 "GET /compute/v2.1/servers/ab4c73d6-9203-4110-93af-4a78428c672b" status: 200 len: 1522 microversion: 2.1 time: 0.393499 Oct 19 00:41:50.275807 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 77/151] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:41:49 2022] GET /compute/v2.1/servers/ab4c73d6-9203-4110-93af-4a78428c672b => generated 1522 bytes in 406 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:41:51.291916 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi [None req-24fba016-7461-4d51-845b-7327b1a10585 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Calling method '>' {{(pid=125640) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:41:51.292537 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.compute.api [None req-24fba016-7461-4d51-845b-7327b1a10585 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] [instance: ab4c73d6-9203-4110-93af-4a78428c672b] Fetching instance by UUID {{(pid=125640) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:41:51.302722 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-24fba016-7461-4d51-845b-7327b1a10585 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:41:51.303317 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-24fba016-7461-4d51-845b-7327b1a10585 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:41:51.303662 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-24fba016-7461-4d51-845b-7327b1a10585 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:41:51.304441 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-24fba016-7461-4d51-845b-7327b1a10585 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:41:51.316940 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-24fba016-7461-4d51-845b-7327b1a10585 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:41:51.319125 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-24fba016-7461-4d51-845b-7327b1a10585 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.013s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:41:51.508526 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.policy [None req-24fba016-7461-4d51-845b-7327b1a10585 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd3f0d615187d4779bdd7da8c9f24c796', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0151d58abd047a692f5cc4e7d05bea5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125640) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:41:51.524304 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: INFO nova.api.openstack.requestlog [None req-24fba016-7461-4d51-845b-7327b1a10585 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] 10.222.0.26 "GET /compute/v2.1/servers/ab4c73d6-9203-4110-93af-4a78428c672b" status: 200 len: 1522 microversion: 2.1 time: 0.236128 Oct 19 00:41:51.524897 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: [pid: 125640|app: 0|req: 75/152] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:41:51 2022] GET /compute/v2.1/servers/ab4c73d6-9203-4110-93af-4a78428c672b => generated 1522 bytes in 237 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:41:52.555562 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-091de119-9764-4f80-973c-f07b01f62d4b tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:41:52.556172 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-091de119-9764-4f80-973c-f07b01f62d4b tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] [instance: ab4c73d6-9203-4110-93af-4a78428c672b] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:41:52.579133 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-091de119-9764-4f80-973c-f07b01f62d4b tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:41:52.579625 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-091de119-9764-4f80-973c-f07b01f62d4b tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:41:52.580050 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-091de119-9764-4f80-973c-f07b01f62d4b tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:41:52.580756 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-091de119-9764-4f80-973c-f07b01f62d4b tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:41:52.581337 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-091de119-9764-4f80-973c-f07b01f62d4b tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:41:52.581618 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-091de119-9764-4f80-973c-f07b01f62d4b tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:41:52.639169 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG dbcounter [-] [125641] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=125641) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 00:41:52.880204 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-091de119-9764-4f80-973c-f07b01f62d4b tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd3f0d615187d4779bdd7da8c9f24c796', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0151d58abd047a692f5cc4e7d05bea5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:41:52.890178 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-091de119-9764-4f80-973c-f07b01f62d4b tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] 10.222.0.26 "GET /compute/v2.1/servers/ab4c73d6-9203-4110-93af-4a78428c672b" status: 200 len: 1522 microversion: 2.1 time: 0.342003 Oct 19 00:41:52.891295 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 78/153] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:41:52 2022] GET /compute/v2.1/servers/ab4c73d6-9203-4110-93af-4a78428c672b => generated 1522 bytes in 345 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:41:53.913530 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi [None req-6f39d467-d9b1-4d71-a590-fe06ff085984 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Calling method '>' {{(pid=125640) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:41:53.914531 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.compute.api [None req-6f39d467-d9b1-4d71-a590-fe06ff085984 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] [instance: ab4c73d6-9203-4110-93af-4a78428c672b] Fetching instance by UUID {{(pid=125640) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:41:53.922710 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-6f39d467-d9b1-4d71-a590-fe06ff085984 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:41:53.924897 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-6f39d467-d9b1-4d71-a590-fe06ff085984 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:41:53.925758 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-6f39d467-d9b1-4d71-a590-fe06ff085984 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:41:53.926534 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-6f39d467-d9b1-4d71-a590-fe06ff085984 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:41:53.926988 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-6f39d467-d9b1-4d71-a590-fe06ff085984 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:41:53.927997 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-6f39d467-d9b1-4d71-a590-fe06ff085984 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:41:54.210727 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.policy [None req-6f39d467-d9b1-4d71-a590-fe06ff085984 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd3f0d615187d4779bdd7da8c9f24c796', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0151d58abd047a692f5cc4e7d05bea5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125640) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:41:54.221380 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: INFO nova.api.openstack.requestlog [None req-6f39d467-d9b1-4d71-a590-fe06ff085984 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] 10.222.0.26 "GET /compute/v2.1/servers/ab4c73d6-9203-4110-93af-4a78428c672b" status: 200 len: 1522 microversion: 2.1 time: 0.311878 Oct 19 00:41:54.235903 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: [pid: 125640|app: 0|req: 76/154] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:41:53 2022] GET /compute/v2.1/servers/ab4c73d6-9203-4110-93af-4a78428c672b => generated 1522 bytes in 327 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:41:55.247431 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-69873557-4da9-489f-854e-2eafa3de9562 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:41:55.249268 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-69873557-4da9-489f-854e-2eafa3de9562 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] [instance: ab4c73d6-9203-4110-93af-4a78428c672b] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:41:55.258442 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-69873557-4da9-489f-854e-2eafa3de9562 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:41:55.259021 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-69873557-4da9-489f-854e-2eafa3de9562 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:41:55.259456 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-69873557-4da9-489f-854e-2eafa3de9562 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:41:55.260150 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-69873557-4da9-489f-854e-2eafa3de9562 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:41:55.260841 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-69873557-4da9-489f-854e-2eafa3de9562 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:41:55.261405 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-69873557-4da9-489f-854e-2eafa3de9562 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:41:55.424698 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-69873557-4da9-489f-854e-2eafa3de9562 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd3f0d615187d4779bdd7da8c9f24c796', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0151d58abd047a692f5cc4e7d05bea5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:41:55.439637 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-69873557-4da9-489f-854e-2eafa3de9562 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] 10.222.0.26 "GET /compute/v2.1/servers/ab4c73d6-9203-4110-93af-4a78428c672b" status: 200 len: 1522 microversion: 2.1 time: 0.193150 Oct 19 00:41:55.452279 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 79/155] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:41:55 2022] GET /compute/v2.1/servers/ab4c73d6-9203-4110-93af-4a78428c672b => generated 1522 bytes in 211 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 19 00:41:56.462959 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi [None req-e5acc8b6-2f4a-4720-a9ae-8af7a07c3b43 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Calling method '>' {{(pid=125640) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:41:56.463867 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.compute.api [None req-e5acc8b6-2f4a-4720-a9ae-8af7a07c3b43 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] [instance: ab4c73d6-9203-4110-93af-4a78428c672b] Fetching instance by UUID {{(pid=125640) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:41:56.474250 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-e5acc8b6-2f4a-4720-a9ae-8af7a07c3b43 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:41:56.474844 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-e5acc8b6-2f4a-4720-a9ae-8af7a07c3b43 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:41:56.475347 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-e5acc8b6-2f4a-4720-a9ae-8af7a07c3b43 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:41:56.476136 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-e5acc8b6-2f4a-4720-a9ae-8af7a07c3b43 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:41:56.476656 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-e5acc8b6-2f4a-4720-a9ae-8af7a07c3b43 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:41:56.477143 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-e5acc8b6-2f4a-4720-a9ae-8af7a07c3b43 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:41:56.621436 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.policy [None req-e5acc8b6-2f4a-4720-a9ae-8af7a07c3b43 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd3f0d615187d4779bdd7da8c9f24c796', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0151d58abd047a692f5cc4e7d05bea5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125640) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:41:56.630659 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: INFO nova.api.openstack.requestlog [None req-e5acc8b6-2f4a-4720-a9ae-8af7a07c3b43 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] 10.222.0.26 "GET /compute/v2.1/servers/ab4c73d6-9203-4110-93af-4a78428c672b" status: 200 len: 1522 microversion: 2.1 time: 0.171628 Oct 19 00:41:56.631166 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: [pid: 125640|app: 0|req: 77/156] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:41:56 2022] GET /compute/v2.1/servers/ab4c73d6-9203-4110-93af-4a78428c672b => generated 1522 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:41:57.655387 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-b6cdf07d-8912-496a-9514-93886bf2e913 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:41:57.657500 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-b6cdf07d-8912-496a-9514-93886bf2e913 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] [instance: ab4c73d6-9203-4110-93af-4a78428c672b] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:41:57.665148 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-b6cdf07d-8912-496a-9514-93886bf2e913 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:41:57.666049 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-b6cdf07d-8912-496a-9514-93886bf2e913 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:41:57.666746 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-b6cdf07d-8912-496a-9514-93886bf2e913 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:41:57.667770 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-b6cdf07d-8912-496a-9514-93886bf2e913 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:41:57.668364 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-b6cdf07d-8912-496a-9514-93886bf2e913 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:41:57.669019 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-b6cdf07d-8912-496a-9514-93886bf2e913 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:41:57.825722 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-b6cdf07d-8912-496a-9514-93886bf2e913 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd3f0d615187d4779bdd7da8c9f24c796', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0151d58abd047a692f5cc4e7d05bea5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:41:57.836489 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-b6cdf07d-8912-496a-9514-93886bf2e913 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] 10.222.0.26 "GET /compute/v2.1/servers/ab4c73d6-9203-4110-93af-4a78428c672b" status: 200 len: 1522 microversion: 2.1 time: 0.185787 Oct 19 00:41:57.837271 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 80/157] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:41:57 2022] GET /compute/v2.1/servers/ab4c73d6-9203-4110-93af-4a78428c672b => generated 1522 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:41:58.861631 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi [None req-b40c175f-011c-40bf-9b7a-e777d53c3127 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Calling method '>' {{(pid=125640) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:41:58.862248 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.compute.api [None req-b40c175f-011c-40bf-9b7a-e777d53c3127 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] [instance: ab4c73d6-9203-4110-93af-4a78428c672b] Fetching instance by UUID {{(pid=125640) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:41:58.868787 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-b40c175f-011c-40bf-9b7a-e777d53c3127 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:41:58.869287 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-b40c175f-011c-40bf-9b7a-e777d53c3127 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:41:58.869724 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-b40c175f-011c-40bf-9b7a-e777d53c3127 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:41:58.870437 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-b40c175f-011c-40bf-9b7a-e777d53c3127 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:41:58.870937 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-b40c175f-011c-40bf-9b7a-e777d53c3127 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:41:58.871366 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-b40c175f-011c-40bf-9b7a-e777d53c3127 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:41:58.876817 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG dbcounter [-] [125640] Writing DB stats nova_cell1:SELECT=81 {{(pid=125640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 00:41:59.009159 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.policy [None req-b40c175f-011c-40bf-9b7a-e777d53c3127 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd3f0d615187d4779bdd7da8c9f24c796', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0151d58abd047a692f5cc4e7d05bea5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125640) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:41:59.018252 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: INFO nova.api.openstack.requestlog [None req-b40c175f-011c-40bf-9b7a-e777d53c3127 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] 10.222.0.26 "GET /compute/v2.1/servers/ab4c73d6-9203-4110-93af-4a78428c672b" status: 200 len: 1522 microversion: 2.1 time: 0.160957 Oct 19 00:41:59.018876 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: [pid: 125640|app: 0|req: 78/158] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:41:58 2022] GET /compute/v2.1/servers/ab4c73d6-9203-4110-93af-4a78428c672b => generated 1522 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:42:00.042383 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-ff613476-6485-4da2-9700-559424aea841 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:42:00.042959 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-ff613476-6485-4da2-9700-559424aea841 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] [instance: ab4c73d6-9203-4110-93af-4a78428c672b] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:42:00.050039 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ff613476-6485-4da2-9700-559424aea841 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:00.050808 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ff613476-6485-4da2-9700-559424aea841 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:00.051242 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ff613476-6485-4da2-9700-559424aea841 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:00.051937 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ff613476-6485-4da2-9700-559424aea841 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:00.052387 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ff613476-6485-4da2-9700-559424aea841 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:00.052805 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ff613476-6485-4da2-9700-559424aea841 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:00.210151 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-ff613476-6485-4da2-9700-559424aea841 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd3f0d615187d4779bdd7da8c9f24c796', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0151d58abd047a692f5cc4e7d05bea5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:42:00.217935 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-ff613476-6485-4da2-9700-559424aea841 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] 10.222.0.26 "GET /compute/v2.1/servers/ab4c73d6-9203-4110-93af-4a78428c672b" status: 200 len: 1522 microversion: 2.1 time: 0.179593 Oct 19 00:42:00.220946 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 81/159] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:42:00 2022] GET /compute/v2.1/servers/ab4c73d6-9203-4110-93af-4a78428c672b => generated 1522 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 19 00:42:01.244142 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi [None req-1120ae87-5312-486f-8a88-483c88adc62f tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Calling method '>' {{(pid=125640) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:42:01.244847 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.compute.api [None req-1120ae87-5312-486f-8a88-483c88adc62f tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] [instance: ab4c73d6-9203-4110-93af-4a78428c672b] Fetching instance by UUID {{(pid=125640) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:42:01.246693 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG dbcounter [-] [125640] Writing DB stats nova_cell0:SELECT=4 {{(pid=125640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 00:42:01.256129 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-1120ae87-5312-486f-8a88-483c88adc62f tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:01.256633 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-1120ae87-5312-486f-8a88-483c88adc62f tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:01.257075 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-1120ae87-5312-486f-8a88-483c88adc62f tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:01.257787 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-1120ae87-5312-486f-8a88-483c88adc62f tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:01.258410 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-1120ae87-5312-486f-8a88-483c88adc62f tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:01.259063 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-1120ae87-5312-486f-8a88-483c88adc62f tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:01.439206 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.policy [None req-1120ae87-5312-486f-8a88-483c88adc62f tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd3f0d615187d4779bdd7da8c9f24c796', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0151d58abd047a692f5cc4e7d05bea5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125640) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:42:01.448317 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: INFO nova.api.openstack.requestlog [None req-1120ae87-5312-486f-8a88-483c88adc62f tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] 10.222.0.26 "GET /compute/v2.1/servers/ab4c73d6-9203-4110-93af-4a78428c672b" status: 200 len: 1522 microversion: 2.1 time: 0.208930 Oct 19 00:42:01.450158 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: [pid: 125640|app: 0|req: 79/160] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:42:01 2022] GET /compute/v2.1/servers/ab4c73d6-9203-4110-93af-4a78428c672b => generated 1522 bytes in 211 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:42:02.103918 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-0934520c-4272-48a1-90c6-57ec8fdfedd6 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesActionsTest-instance-1326143625", "imageRef": "b387b4f8-f75e-4a68-b49c-a015bf61c46b", "flavorRef": "452", "networks": [{"uuid": "7d7558f7-ec39-4fba-83f7-3ca748451727"}]}} {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 19 00:42:02.168676 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.network.neutron [None req-0934520c-4272-48a1-90c6-57ec8fdfedd6 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] validate_networks() for [('7d7558f7-ec39-4fba-83f7-3ca748451727', None, None, None, None, None)] {{(pid=125641) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 19 00:42:02.473466 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi [None req-4c06a527-3975-4959-9910-39458f157e1d tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Calling method '>' {{(pid=125640) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:42:02.474064 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.compute.api [None req-4c06a527-3975-4959-9910-39458f157e1d tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] [instance: ab4c73d6-9203-4110-93af-4a78428c672b] Fetching instance by UUID {{(pid=125640) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:42:02.480623 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-4c06a527-3975-4959-9910-39458f157e1d tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:02.481106 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-4c06a527-3975-4959-9910-39458f157e1d tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:02.481569 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-4c06a527-3975-4959-9910-39458f157e1d tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:02.482271 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-4c06a527-3975-4959-9910-39458f157e1d tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:02.482757 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-4c06a527-3975-4959-9910-39458f157e1d tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:02.483656 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-4c06a527-3975-4959-9910-39458f157e1d tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:02.532670 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.virt.hardware [None req-0934520c-4272-48a1-90c6-57ec8fdfedd6 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Flavor limits 0:0:0 {{(pid=125641) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 19 00:42:02.533092 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.virt.hardware [None req-0934520c-4272-48a1-90c6-57ec8fdfedd6 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Image limits 0:0:0 {{(pid=125641) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 19 00:42:02.533568 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.virt.hardware [None req-0934520c-4272-48a1-90c6-57ec8fdfedd6 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Flavor pref 0:0:0 {{(pid=125641) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 19 00:42:02.533974 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.virt.hardware [None req-0934520c-4272-48a1-90c6-57ec8fdfedd6 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Image pref 0:0:0 {{(pid=125641) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 19 00:42:02.534387 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.virt.hardware [None req-0934520c-4272-48a1-90c6-57ec8fdfedd6 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=125641) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 19 00:42:02.550126 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-0934520c-4272-48a1-90c6-57ec8fdfedd6 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:02.551108 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-0934520c-4272-48a1-90c6-57ec8fdfedd6 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:02.551770 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-0934520c-4272-48a1-90c6-57ec8fdfedd6 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:02.552859 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-0934520c-4272-48a1-90c6-57ec8fdfedd6 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:02.553481 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-0934520c-4272-48a1-90c6-57ec8fdfedd6 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:02.554128 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-0934520c-4272-48a1-90c6-57ec8fdfedd6 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:02.574592 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.quota [None req-0934520c-4272-48a1-90c6-57ec8fdfedd6 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Getting quotas for project 173a52b79dab4962a4aa380c26f97d2d. Resources: {'cores', 'ram', 'instances'} {{(pid=125641) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 19 00:42:02.583221 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.quota [None req-0934520c-4272-48a1-90c6-57ec8fdfedd6 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Getting quotas for user a08c9457231a447eb1c4130e68cfb31d and project 173a52b79dab4962a4aa380c26f97d2d. Resources: {'cores', 'ram', 'instances'} {{(pid=125641) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 19 00:42:02.600399 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-0934520c-4272-48a1-90c6-57ec8fdfedd6 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Going to run 1 instances... {{(pid=125641) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 19 00:42:02.602870 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-0934520c-4272-48a1-90c6-57ec8fdfedd6 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] [instance: 66990043-23b4-4151-8137-6637e4b0a1db] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='b387b4f8-f75e-4a68-b49c-a015bf61c46b',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=125641) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 19 00:42:02.641326 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.policy [None req-4c06a527-3975-4959-9910-39458f157e1d tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd3f0d615187d4779bdd7da8c9f24c796', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0151d58abd047a692f5cc4e7d05bea5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125640) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:42:02.651000 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-0934520c-4272-48a1-90c6-57ec8fdfedd6 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.550602 Oct 19 00:42:02.651848 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 82/161] 10.222.0.26 () {60 vars in 1147 bytes} [Wed Oct 19 00:42:02 2022] POST /compute/v2.1/servers => generated 380 bytes in 552 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 19 00:42:02.654752 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: INFO nova.api.openstack.requestlog [None req-4c06a527-3975-4959-9910-39458f157e1d tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] 10.222.0.26 "GET /compute/v2.1/servers/ab4c73d6-9203-4110-93af-4a78428c672b" status: 200 len: 1522 microversion: 2.1 time: 0.185542 Oct 19 00:42:02.655324 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: [pid: 125640|app: 0|req: 80/162] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:42:02 2022] GET /compute/v2.1/servers/ab4c73d6-9203-4110-93af-4a78428c672b => generated 1522 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 19 00:42:02.671008 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-48a995a0-59d4-49e5-a39b-5ef47a465c74 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:42:02.672670 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-48a995a0-59d4-49e5-a39b-5ef47a465c74 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] [instance: 66990043-23b4-4151-8137-6637e4b0a1db] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:42:02.737259 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-48a995a0-59d4-49e5-a39b-5ef47a465c74 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a08c9457231a447eb1c4130e68cfb31d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '173a52b79dab4962a4aa380c26f97d2d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:42:02.749765 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-48a995a0-59d4-49e5-a39b-5ef47a465c74 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] 10.222.0.26 "GET /compute/v2.1/servers/66990043-23b4-4151-8137-6637e4b0a1db" status: 200 len: 1214 microversion: 2.1 time: 0.085668 Oct 19 00:42:02.751486 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 83/163] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:42:02 2022] GET /compute/v2.1/servers/66990043-23b4-4151-8137-6637e4b0a1db => generated 1214 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:42:03.678545 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi [None req-bff4bac7-b930-45ce-b791-1b6c24e8009a tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Calling method '>' {{(pid=125640) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:42:03.679160 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.compute.api [None req-bff4bac7-b930-45ce-b791-1b6c24e8009a tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] [instance: ab4c73d6-9203-4110-93af-4a78428c672b] Fetching instance by UUID {{(pid=125640) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:42:03.689019 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-bff4bac7-b930-45ce-b791-1b6c24e8009a tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:03.689512 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-bff4bac7-b930-45ce-b791-1b6c24e8009a tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:03.690044 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-bff4bac7-b930-45ce-b791-1b6c24e8009a tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:03.690833 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-bff4bac7-b930-45ce-b791-1b6c24e8009a tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:03.691350 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-bff4bac7-b930-45ce-b791-1b6c24e8009a tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:03.691794 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-bff4bac7-b930-45ce-b791-1b6c24e8009a tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:03.776806 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-fe676a3b-fbd0-4e54-be87-b515bc176d47 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:42:03.777210 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-fe676a3b-fbd0-4e54-be87-b515bc176d47 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] [instance: 66990043-23b4-4151-8137-6637e4b0a1db] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:42:03.783946 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-fe676a3b-fbd0-4e54-be87-b515bc176d47 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:03.784426 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-fe676a3b-fbd0-4e54-be87-b515bc176d47 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:03.784859 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-fe676a3b-fbd0-4e54-be87-b515bc176d47 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:03.785546 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-fe676a3b-fbd0-4e54-be87-b515bc176d47 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:03.785994 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-fe676a3b-fbd0-4e54-be87-b515bc176d47 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:03.786412 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-fe676a3b-fbd0-4e54-be87-b515bc176d47 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:03.850657 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.policy [None req-bff4bac7-b930-45ce-b791-1b6c24e8009a tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd3f0d615187d4779bdd7da8c9f24c796', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0151d58abd047a692f5cc4e7d05bea5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125640) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:42:03.857446 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-fe676a3b-fbd0-4e54-be87-b515bc176d47 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a08c9457231a447eb1c4130e68cfb31d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '173a52b79dab4962a4aa380c26f97d2d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:42:03.858530 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: INFO nova.api.openstack.requestlog [None req-bff4bac7-b930-45ce-b791-1b6c24e8009a tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] 10.222.0.26 "GET /compute/v2.1/servers/ab4c73d6-9203-4110-93af-4a78428c672b" status: 200 len: 1502 microversion: 2.1 time: 0.184506 Oct 19 00:42:03.859995 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: [pid: 125640|app: 0|req: 81/164] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:42:03 2022] GET /compute/v2.1/servers/ab4c73d6-9203-4110-93af-4a78428c672b => generated 1502 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:42:03.866167 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-fe676a3b-fbd0-4e54-be87-b515bc176d47 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] 10.222.0.26 "GET /compute/v2.1/servers/66990043-23b4-4151-8137-6637e4b0a1db" status: 200 len: 1266 microversion: 2.1 time: 0.093406 Oct 19 00:42:03.866850 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 84/165] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:42:03 2022] GET /compute/v2.1/servers/66990043-23b4-4151-8137-6637e4b0a1db => generated 1266 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 19 00:42:03.883619 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi [None req-6520e409-05a3-4a9e-90c5-90c833bf456f tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Calling method '>' {{(pid=125640) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:42:03.884242 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.compute.api [None req-6520e409-05a3-4a9e-90c5-90c833bf456f tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] [instance: 3da61834-fd67-49d2-925c-c18fc264042c] Fetching instance by UUID {{(pid=125640) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:42:03.890512 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-6520e409-05a3-4a9e-90c5-90c833bf456f tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:03.891004 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-6520e409-05a3-4a9e-90c5-90c833bf456f tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:03.891447 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-6520e409-05a3-4a9e-90c5-90c833bf456f tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:03.892149 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-6520e409-05a3-4a9e-90c5-90c833bf456f tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:03.892622 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-6520e409-05a3-4a9e-90c5-90c833bf456f tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:03.893046 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-6520e409-05a3-4a9e-90c5-90c833bf456f tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:04.033235 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.policy [None req-6520e409-05a3-4a9e-90c5-90c833bf456f tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd3f0d615187d4779bdd7da8c9f24c796', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0151d58abd047a692f5cc4e7d05bea5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125640) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:42:04.041089 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: INFO nova.api.openstack.requestlog [None req-6520e409-05a3-4a9e-90c5-90c833bf456f tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] 10.222.0.26 "GET /compute/v2.1/servers/3da61834-fd67-49d2-925c-c18fc264042c" status: 200 len: 1515 microversion: 2.1 time: 0.163105 Oct 19 00:42:04.041723 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: [pid: 125640|app: 0|req: 82/166] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:42:03 2022] GET /compute/v2.1/servers/3da61834-fd67-49d2-925c-c18fc264042c => generated 1515 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:42:04.890706 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-8f770f3f-3e15-4c35-90bb-1ef8a385df0a tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:42:04.891337 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-8f770f3f-3e15-4c35-90bb-1ef8a385df0a tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] [instance: 66990043-23b4-4151-8137-6637e4b0a1db] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:42:04.897436 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-8f770f3f-3e15-4c35-90bb-1ef8a385df0a tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:04.897998 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-8f770f3f-3e15-4c35-90bb-1ef8a385df0a tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:04.898462 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-8f770f3f-3e15-4c35-90bb-1ef8a385df0a tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:04.899221 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-8f770f3f-3e15-4c35-90bb-1ef8a385df0a tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:04.899718 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-8f770f3f-3e15-4c35-90bb-1ef8a385df0a tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:04.900166 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-8f770f3f-3e15-4c35-90bb-1ef8a385df0a tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:04.962758 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-8f770f3f-3e15-4c35-90bb-1ef8a385df0a tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a08c9457231a447eb1c4130e68cfb31d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '173a52b79dab4962a4aa380c26f97d2d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:42:04.971149 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-8f770f3f-3e15-4c35-90bb-1ef8a385df0a tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] 10.222.0.26 "GET /compute/v2.1/servers/66990043-23b4-4151-8137-6637e4b0a1db" status: 200 len: 1266 microversion: 2.1 time: 0.084776 Oct 19 00:42:04.971740 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 85/167] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:42:04 2022] GET /compute/v2.1/servers/66990043-23b4-4151-8137-6637e4b0a1db => generated 1266 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:42:05.995518 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi [None req-a2f204af-4d2e-4d0a-b259-be2d6eff49ae tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Calling method '>' {{(pid=125640) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:42:05.996041 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.compute.api [None req-a2f204af-4d2e-4d0a-b259-be2d6eff49ae tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] [instance: 66990043-23b4-4151-8137-6637e4b0a1db] Fetching instance by UUID {{(pid=125640) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:42:05.998313 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG dbcounter [-] [125640] Writing DB stats nova_api:SELECT=45,nova_api:INSERT=1 {{(pid=125640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 00:42:06.004322 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-a2f204af-4d2e-4d0a-b259-be2d6eff49ae tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:06.004834 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-a2f204af-4d2e-4d0a-b259-be2d6eff49ae tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:06.005292 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-a2f204af-4d2e-4d0a-b259-be2d6eff49ae tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:06.005991 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-a2f204af-4d2e-4d0a-b259-be2d6eff49ae tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:06.006448 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-a2f204af-4d2e-4d0a-b259-be2d6eff49ae tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:06.006897 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-a2f204af-4d2e-4d0a-b259-be2d6eff49ae tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:06.071188 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.policy [None req-a2f204af-4d2e-4d0a-b259-be2d6eff49ae tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a08c9457231a447eb1c4130e68cfb31d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '173a52b79dab4962a4aa380c26f97d2d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125640) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:42:06.080196 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: INFO nova.api.openstack.requestlog [None req-a2f204af-4d2e-4d0a-b259-be2d6eff49ae tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] 10.222.0.26 "GET /compute/v2.1/servers/66990043-23b4-4151-8137-6637e4b0a1db" status: 200 len: 1266 microversion: 2.1 time: 0.088534 Oct 19 00:42:06.080779 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: [pid: 125640|app: 0|req: 83/168] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:42:05 2022] GET /compute/v2.1/servers/66990043-23b4-4151-8137-6637e4b0a1db => generated 1266 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:42:07.103801 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-ce1ed9e0-be73-44ac-a0e8-779fb26e5ec2 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:42:07.104417 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-ce1ed9e0-be73-44ac-a0e8-779fb26e5ec2 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] [instance: 66990043-23b4-4151-8137-6637e4b0a1db] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:42:07.106319 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG dbcounter [-] [125641] Writing DB stats nova_api:SELECT=51,nova_api:INSERT=3 {{(pid=125641) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 00:42:07.111995 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ce1ed9e0-be73-44ac-a0e8-779fb26e5ec2 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:07.112480 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ce1ed9e0-be73-44ac-a0e8-779fb26e5ec2 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:07.112918 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ce1ed9e0-be73-44ac-a0e8-779fb26e5ec2 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:07.113611 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ce1ed9e0-be73-44ac-a0e8-779fb26e5ec2 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:07.114065 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ce1ed9e0-be73-44ac-a0e8-779fb26e5ec2 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:07.114507 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ce1ed9e0-be73-44ac-a0e8-779fb26e5ec2 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:07.178656 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-ce1ed9e0-be73-44ac-a0e8-779fb26e5ec2 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a08c9457231a447eb1c4130e68cfb31d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '173a52b79dab4962a4aa380c26f97d2d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:42:07.186740 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-ce1ed9e0-be73-44ac-a0e8-779fb26e5ec2 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] 10.222.0.26 "GET /compute/v2.1/servers/66990043-23b4-4151-8137-6637e4b0a1db" status: 200 len: 1272 microversion: 2.1 time: 0.086783 Oct 19 00:42:07.187905 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 86/169] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:42:07 2022] GET /compute/v2.1/servers/66990043-23b4-4151-8137-6637e4b0a1db => generated 1272 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:42:07.584065 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi [None req-8bca9c45-3af8-4b86-abed-a8dc47e58c09 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Action: 'action', calling method: >, body: {"rescue": {"adminPass": "***"}} {{(pid=125640) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 19 00:42:07.586616 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.compute.api [None req-8bca9c45-3af8-4b86-abed-a8dc47e58c09 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] [instance: 3da61834-fd67-49d2-925c-c18fc264042c] Fetching instance by UUID {{(pid=125640) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:42:07.593744 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-8bca9c45-3af8-4b86-abed-a8dc47e58c09 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:07.594234 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-8bca9c45-3af8-4b86-abed-a8dc47e58c09 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:07.594913 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-8bca9c45-3af8-4b86-abed-a8dc47e58c09 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:07.596079 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-8bca9c45-3af8-4b86-abed-a8dc47e58c09 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:07.596538 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-8bca9c45-3af8-4b86-abed-a8dc47e58c09 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:07.596973 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-8bca9c45-3af8-4b86-abed-a8dc47e58c09 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:07.625140 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.objects.instance [None req-8bca9c45-3af8-4b86-abed-a8dc47e58c09 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lazy-loading 'flavor' on Instance uuid 3da61834-fd67-49d2-925c-c18fc264042c {{(pid=125640) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 19 00:42:07.716729 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: INFO nova.api.openstack.requestlog [None req-8bca9c45-3af8-4b86-abed-a8dc47e58c09 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] 10.222.0.26 "POST /compute/v2.1/servers/3da61834-fd67-49d2-925c-c18fc264042c/action" status: 200 len: 32 microversion: 2.1 time: 0.136654 Oct 19 00:42:07.717729 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: [pid: 125640|app: 0|req: 84/170] 10.222.0.26 () {60 vars in 1278 bytes} [Wed Oct 19 00:42:07 2022] POST /compute/v2.1/servers/3da61834-fd67-49d2-925c-c18fc264042c/action => generated 32 bytes in 138 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Oct 19 00:42:07.728673 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-6eb6ccdf-3378-48b0-95fa-1b2da49513d6 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:42:07.729610 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-6eb6ccdf-3378-48b0-95fa-1b2da49513d6 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] [instance: 3da61834-fd67-49d2-925c-c18fc264042c] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:42:07.741656 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-6eb6ccdf-3378-48b0-95fa-1b2da49513d6 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:07.742162 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-6eb6ccdf-3378-48b0-95fa-1b2da49513d6 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:07.742634 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-6eb6ccdf-3378-48b0-95fa-1b2da49513d6 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:07.746342 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-6eb6ccdf-3378-48b0-95fa-1b2da49513d6 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:07.746857 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-6eb6ccdf-3378-48b0-95fa-1b2da49513d6 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:07.747304 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-6eb6ccdf-3378-48b0-95fa-1b2da49513d6 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:08.115966 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-6eb6ccdf-3378-48b0-95fa-1b2da49513d6 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd3f0d615187d4779bdd7da8c9f24c796', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0151d58abd047a692f5cc4e7d05bea5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:42:08.125098 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-6eb6ccdf-3378-48b0-95fa-1b2da49513d6 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] 10.222.0.26 "GET /compute/v2.1/servers/3da61834-fd67-49d2-925c-c18fc264042c" status: 200 len: 1521 microversion: 2.1 time: 0.400177 Oct 19 00:42:08.130118 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 87/171] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:42:07 2022] GET /compute/v2.1/servers/3da61834-fd67-49d2-925c-c18fc264042c => generated 1521 bytes in 407 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:42:08.212202 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi [None req-4149e658-83ca-4ab0-8b97-37fa7718075a tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Calling method '>' {{(pid=125640) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:42:08.213062 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.compute.api [None req-4149e658-83ca-4ab0-8b97-37fa7718075a tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] [instance: 66990043-23b4-4151-8137-6637e4b0a1db] Fetching instance by UUID {{(pid=125640) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:42:08.223168 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-4149e658-83ca-4ab0-8b97-37fa7718075a tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:08.224053 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-4149e658-83ca-4ab0-8b97-37fa7718075a tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:08.224728 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-4149e658-83ca-4ab0-8b97-37fa7718075a tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:08.225673 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-4149e658-83ca-4ab0-8b97-37fa7718075a tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:08.226358 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-4149e658-83ca-4ab0-8b97-37fa7718075a tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:08.227047 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-4149e658-83ca-4ab0-8b97-37fa7718075a tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:08.584479 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.policy [None req-4149e658-83ca-4ab0-8b97-37fa7718075a tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a08c9457231a447eb1c4130e68cfb31d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '173a52b79dab4962a4aa380c26f97d2d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125640) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:42:08.595475 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: INFO nova.api.openstack.requestlog [None req-4149e658-83ca-4ab0-8b97-37fa7718075a tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] 10.222.0.26 "GET /compute/v2.1/servers/66990043-23b4-4151-8137-6637e4b0a1db" status: 200 len: 1314 microversion: 2.1 time: 0.388773 Oct 19 00:42:08.596157 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: [pid: 125640|app: 0|req: 85/172] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:42:08 2022] GET /compute/v2.1/servers/66990043-23b4-4151-8137-6637e4b0a1db => generated 1314 bytes in 390 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:42:08.786114 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [req-961fec54-ff95-40da-8d48-40841245f008 req-0a422247-43b3-4bdb-a082-e5ccef13cef6 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "66990043-23b4-4151-8137-6637e4b0a1db", "tag": "4d3e0aaa-9ce0-44ac-86e1-3ac2ffbc0ec1"}]} {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 19 00:42:08.798476 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-961fec54-ff95-40da-8d48-40841245f008 req-0a422247-43b3-4bdb-a082-e5ccef13cef6 service nova] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:08.799337 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-961fec54-ff95-40da-8d48-40841245f008 req-0a422247-43b3-4bdb-a082-e5ccef13cef6 service nova] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:08.800012 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-961fec54-ff95-40da-8d48-40841245f008 req-0a422247-43b3-4bdb-a082-e5ccef13cef6 service nova] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:08.821182 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.compute.server_external_events [req-961fec54-ff95-40da-8d48-40841245f008 req-0a422247-43b3-4bdb-a082-e5ccef13cef6 service nova] Creating event network-changed:4d3e0aaa-9ce0-44ac-86e1-3ac2ffbc0ec1 for instance 66990043-23b4-4151-8137-6637e4b0a1db on hv-cinder-maste Oct 19 00:42:08.827965 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [req-961fec54-ff95-40da-8d48-40841245f008 req-0a422247-43b3-4bdb-a082-e5ccef13cef6 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.045620 Oct 19 00:42:08.828907 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 88/173] 10.222.0.26 () {64 vars in 1317 bytes} [Wed Oct 19 00:42:08 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 19 00:42:09.157300 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi [None req-19c4c0f0-66ad-42b8-9eb2-9c55223f7e73 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Calling method '>' {{(pid=125640) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:42:09.158276 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.compute.api [None req-19c4c0f0-66ad-42b8-9eb2-9c55223f7e73 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] [instance: 3da61834-fd67-49d2-925c-c18fc264042c] Fetching instance by UUID {{(pid=125640) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:42:09.184866 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-19c4c0f0-66ad-42b8-9eb2-9c55223f7e73 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:09.185367 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-19c4c0f0-66ad-42b8-9eb2-9c55223f7e73 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:09.185864 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-19c4c0f0-66ad-42b8-9eb2-9c55223f7e73 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:09.190513 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-19c4c0f0-66ad-42b8-9eb2-9c55223f7e73 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:09.191021 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-19c4c0f0-66ad-42b8-9eb2-9c55223f7e73 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:09.191464 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-19c4c0f0-66ad-42b8-9eb2-9c55223f7e73 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:09.451713 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.policy [None req-19c4c0f0-66ad-42b8-9eb2-9c55223f7e73 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd3f0d615187d4779bdd7da8c9f24c796', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0151d58abd047a692f5cc4e7d05bea5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125640) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:42:09.459752 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: INFO nova.api.openstack.requestlog [None req-19c4c0f0-66ad-42b8-9eb2-9c55223f7e73 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] 10.222.0.26 "GET /compute/v2.1/servers/3da61834-fd67-49d2-925c-c18fc264042c" status: 200 len: 1521 microversion: 2.1 time: 0.309853 Oct 19 00:42:09.460331 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: [pid: 125640|app: 0|req: 86/174] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:42:09 2022] GET /compute/v2.1/servers/3da61834-fd67-49d2-925c-c18fc264042c => generated 1521 bytes in 311 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:42:09.620247 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-23bb73fd-42f8-4746-8eb7-ddbc780c2804 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:42:09.621110 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-23bb73fd-42f8-4746-8eb7-ddbc780c2804 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] [instance: 66990043-23b4-4151-8137-6637e4b0a1db] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:42:09.627619 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-23bb73fd-42f8-4746-8eb7-ddbc780c2804 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:09.628500 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-23bb73fd-42f8-4746-8eb7-ddbc780c2804 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:09.628623 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-23bb73fd-42f8-4746-8eb7-ddbc780c2804 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:09.629227 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-23bb73fd-42f8-4746-8eb7-ddbc780c2804 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:09.629755 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-23bb73fd-42f8-4746-8eb7-ddbc780c2804 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:09.630202 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-23bb73fd-42f8-4746-8eb7-ddbc780c2804 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:09.775022 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-23bb73fd-42f8-4746-8eb7-ddbc780c2804 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a08c9457231a447eb1c4130e68cfb31d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '173a52b79dab4962a4aa380c26f97d2d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:42:09.785400 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-23bb73fd-42f8-4746-8eb7-ddbc780c2804 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] 10.222.0.26 "GET /compute/v2.1/servers/66990043-23b4-4151-8137-6637e4b0a1db" status: 200 len: 1314 microversion: 2.1 time: 0.169262 Oct 19 00:42:09.786341 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 89/175] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:42:09 2022] GET /compute/v2.1/servers/66990043-23b4-4151-8137-6637e4b0a1db => generated 1314 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:42:10.483076 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi [None req-b87089c3-b9ae-4f09-b44f-707337b42197 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Calling method '>' {{(pid=125640) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:42:10.483797 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.compute.api [None req-b87089c3-b9ae-4f09-b44f-707337b42197 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] [instance: 3da61834-fd67-49d2-925c-c18fc264042c] Fetching instance by UUID {{(pid=125640) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:42:10.490542 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-b87089c3-b9ae-4f09-b44f-707337b42197 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:10.491037 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-b87089c3-b9ae-4f09-b44f-707337b42197 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:10.491506 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-b87089c3-b9ae-4f09-b44f-707337b42197 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:10.492198 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-b87089c3-b9ae-4f09-b44f-707337b42197 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:10.492657 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-b87089c3-b9ae-4f09-b44f-707337b42197 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:10.493075 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-b87089c3-b9ae-4f09-b44f-707337b42197 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:10.700215 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.policy [None req-b87089c3-b9ae-4f09-b44f-707337b42197 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd3f0d615187d4779bdd7da8c9f24c796', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0151d58abd047a692f5cc4e7d05bea5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125640) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:42:10.708832 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: INFO nova.api.openstack.requestlog [None req-b87089c3-b9ae-4f09-b44f-707337b42197 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] 10.222.0.26 "GET /compute/v2.1/servers/3da61834-fd67-49d2-925c-c18fc264042c" status: 200 len: 1521 microversion: 2.1 time: 0.229530 Oct 19 00:42:10.709604 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: [pid: 125640|app: 0|req: 87/176] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:42:10 2022] GET /compute/v2.1/servers/3da61834-fd67-49d2-925c-c18fc264042c => generated 1521 bytes in 231 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:42:10.810870 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-0ec2b97c-ee1c-4210-91c8-113adb205f84 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:42:10.811462 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-0ec2b97c-ee1c-4210-91c8-113adb205f84 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] [instance: 66990043-23b4-4151-8137-6637e4b0a1db] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:42:10.818625 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-0ec2b97c-ee1c-4210-91c8-113adb205f84 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:10.819768 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-0ec2b97c-ee1c-4210-91c8-113adb205f84 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:10.820292 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-0ec2b97c-ee1c-4210-91c8-113adb205f84 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:10.821010 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-0ec2b97c-ee1c-4210-91c8-113adb205f84 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:10.821463 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-0ec2b97c-ee1c-4210-91c8-113adb205f84 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:10.821878 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-0ec2b97c-ee1c-4210-91c8-113adb205f84 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:10.964474 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-0ec2b97c-ee1c-4210-91c8-113adb205f84 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a08c9457231a447eb1c4130e68cfb31d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '173a52b79dab4962a4aa380c26f97d2d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:42:10.973752 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-0ec2b97c-ee1c-4210-91c8-113adb205f84 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] 10.222.0.26 "GET /compute/v2.1/servers/66990043-23b4-4151-8137-6637e4b0a1db" status: 200 len: 1314 microversion: 2.1 time: 0.168260 Oct 19 00:42:10.974411 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 90/177] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:42:10 2022] GET /compute/v2.1/servers/66990043-23b4-4151-8137-6637e4b0a1db => generated 1314 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:42:11.735784 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi [None req-1cff0113-242e-44aa-a233-21bcb9ed86f3 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Calling method '>' {{(pid=125640) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:42:11.736480 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.compute.api [None req-1cff0113-242e-44aa-a233-21bcb9ed86f3 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] [instance: 3da61834-fd67-49d2-925c-c18fc264042c] Fetching instance by UUID {{(pid=125640) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:42:11.743133 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-1cff0113-242e-44aa-a233-21bcb9ed86f3 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:11.743643 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-1cff0113-242e-44aa-a233-21bcb9ed86f3 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:11.744288 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-1cff0113-242e-44aa-a233-21bcb9ed86f3 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:11.745016 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-1cff0113-242e-44aa-a233-21bcb9ed86f3 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:11.745484 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-1cff0113-242e-44aa-a233-21bcb9ed86f3 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:11.745930 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-1cff0113-242e-44aa-a233-21bcb9ed86f3 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:11.882451 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.policy [None req-1cff0113-242e-44aa-a233-21bcb9ed86f3 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd3f0d615187d4779bdd7da8c9f24c796', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0151d58abd047a692f5cc4e7d05bea5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125640) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:42:11.891070 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: INFO nova.api.openstack.requestlog [None req-1cff0113-242e-44aa-a233-21bcb9ed86f3 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] 10.222.0.26 "GET /compute/v2.1/servers/3da61834-fd67-49d2-925c-c18fc264042c" status: 200 len: 1521 microversion: 2.1 time: 0.160115 Oct 19 00:42:11.891646 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: [pid: 125640|app: 0|req: 88/178] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:42:11 2022] GET /compute/v2.1/servers/3da61834-fd67-49d2-925c-c18fc264042c => generated 1521 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:42:11.996333 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-9c17401c-a875-46c8-b332-1052b869cd07 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:42:11.996941 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-9c17401c-a875-46c8-b332-1052b869cd07 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] [instance: 66990043-23b4-4151-8137-6637e4b0a1db] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:42:12.003545 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-9c17401c-a875-46c8-b332-1052b869cd07 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:12.004029 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-9c17401c-a875-46c8-b332-1052b869cd07 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:12.004468 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-9c17401c-a875-46c8-b332-1052b869cd07 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:12.005158 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-9c17401c-a875-46c8-b332-1052b869cd07 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:12.005608 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-9c17401c-a875-46c8-b332-1052b869cd07 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:12.006025 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-9c17401c-a875-46c8-b332-1052b869cd07 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:12.145326 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-9c17401c-a875-46c8-b332-1052b869cd07 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a08c9457231a447eb1c4130e68cfb31d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '173a52b79dab4962a4aa380c26f97d2d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:42:12.153489 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-9c17401c-a875-46c8-b332-1052b869cd07 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] 10.222.0.26 "GET /compute/v2.1/servers/66990043-23b4-4151-8137-6637e4b0a1db" status: 200 len: 1314 microversion: 2.1 time: 0.160834 Oct 19 00:42:12.154953 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 91/179] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:42:11 2022] GET /compute/v2.1/servers/66990043-23b4-4151-8137-6637e4b0a1db => generated 1314 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:42:12.915199 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi [None req-4b14b8d8-c53d-4af2-bd09-da4833178edc tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Calling method '>' {{(pid=125640) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:42:12.915811 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.compute.api [None req-4b14b8d8-c53d-4af2-bd09-da4833178edc tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] [instance: 3da61834-fd67-49d2-925c-c18fc264042c] Fetching instance by UUID {{(pid=125640) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:42:12.922568 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-4b14b8d8-c53d-4af2-bd09-da4833178edc tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:12.923049 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-4b14b8d8-c53d-4af2-bd09-da4833178edc tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:12.923478 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-4b14b8d8-c53d-4af2-bd09-da4833178edc tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:12.924161 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-4b14b8d8-c53d-4af2-bd09-da4833178edc tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:12.924615 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-4b14b8d8-c53d-4af2-bd09-da4833178edc tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:12.925035 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-4b14b8d8-c53d-4af2-bd09-da4833178edc tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:13.054943 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.policy [None req-4b14b8d8-c53d-4af2-bd09-da4833178edc tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd3f0d615187d4779bdd7da8c9f24c796', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0151d58abd047a692f5cc4e7d05bea5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125640) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:42:13.065215 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: INFO nova.api.openstack.requestlog [None req-4b14b8d8-c53d-4af2-bd09-da4833178edc tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] 10.222.0.26 "GET /compute/v2.1/servers/3da61834-fd67-49d2-925c-c18fc264042c" status: 200 len: 1521 microversion: 2.1 time: 0.154634 Oct 19 00:42:13.065799 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: [pid: 125640|app: 0|req: 89/180] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:42:12 2022] GET /compute/v2.1/servers/3da61834-fd67-49d2-925c-c18fc264042c => generated 1521 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 19 00:42:13.178096 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-02cd59ee-8164-4640-9469-127aeb0c63dc tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:42:13.178771 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-02cd59ee-8164-4640-9469-127aeb0c63dc tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] [instance: 66990043-23b4-4151-8137-6637e4b0a1db] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:42:13.181838 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG dbcounter [-] [125641] Writing DB stats nova_cell0:SELECT=2 {{(pid=125641) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 00:42:13.182700 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG dbcounter [-] [125641] Writing DB stats nova_cell0:SELECT=8 {{(pid=125641) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 00:42:13.188436 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-02cd59ee-8164-4640-9469-127aeb0c63dc tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:13.188917 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-02cd59ee-8164-4640-9469-127aeb0c63dc tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:13.189352 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-02cd59ee-8164-4640-9469-127aeb0c63dc tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:13.190052 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-02cd59ee-8164-4640-9469-127aeb0c63dc tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:13.190529 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-02cd59ee-8164-4640-9469-127aeb0c63dc tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:13.190954 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-02cd59ee-8164-4640-9469-127aeb0c63dc tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:13.336421 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-02cd59ee-8164-4640-9469-127aeb0c63dc tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a08c9457231a447eb1c4130e68cfb31d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '173a52b79dab4962a4aa380c26f97d2d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:42:13.344286 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-02cd59ee-8164-4640-9469-127aeb0c63dc tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] 10.222.0.26 "GET /compute/v2.1/servers/66990043-23b4-4151-8137-6637e4b0a1db" status: 200 len: 1314 microversion: 2.1 time: 0.170022 Oct 19 00:42:13.344872 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 92/181] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:42:13 2022] GET /compute/v2.1/servers/66990043-23b4-4151-8137-6637e4b0a1db => generated 1314 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:42:14.063009 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi [req-e05e1993-e45b-4518-82ac-6853426c931a req-0b235259-39b5-4fd2-84e8-80b1a8b72669 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "66990043-23b4-4151-8137-6637e4b0a1db", "name": "network-vif-plugged", "status": "completed", "tag": "4d3e0aaa-9ce0-44ac-86e1-3ac2ffbc0ec1"}]} {{(pid=125640) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 19 00:42:14.075604 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [req-e05e1993-e45b-4518-82ac-6853426c931a req-0b235259-39b5-4fd2-84e8-80b1a8b72669 service nova] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:14.076090 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [req-e05e1993-e45b-4518-82ac-6853426c931a req-0b235259-39b5-4fd2-84e8-80b1a8b72669 service nova] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:14.076555 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [req-e05e1993-e45b-4518-82ac-6853426c931a req-0b235259-39b5-4fd2-84e8-80b1a8b72669 service nova] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:14.088860 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-4a61f401-1e7c-469e-a566-a6d7a8339d19 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:42:14.089859 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-4a61f401-1e7c-469e-a566-a6d7a8339d19 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] [instance: 3da61834-fd67-49d2-925c-c18fc264042c] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:42:14.090271 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: INFO nova.api.openstack.compute.server_external_events [req-e05e1993-e45b-4518-82ac-6853426c931a req-0b235259-39b5-4fd2-84e8-80b1a8b72669 service nova] Creating event network-vif-plugged:4d3e0aaa-9ce0-44ac-86e1-3ac2ffbc0ec1 for instance 66990043-23b4-4151-8137-6637e4b0a1db on hv-cinder-maste Oct 19 00:42:14.098322 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: INFO nova.api.openstack.requestlog [req-e05e1993-e45b-4518-82ac-6853426c931a req-0b235259-39b5-4fd2-84e8-80b1a8b72669 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.040340 Oct 19 00:42:14.098926 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-4a61f401-1e7c-469e-a566-a6d7a8339d19 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:14.099643 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-4a61f401-1e7c-469e-a566-a6d7a8339d19 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:14.099802 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: [pid: 125640|app: 0|req: 90/182] 10.222.0.26 () {64 vars in 1317 bytes} [Wed Oct 19 00:42:14 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 19 00:42:14.100378 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-4a61f401-1e7c-469e-a566-a6d7a8339d19 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:14.101712 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-4a61f401-1e7c-469e-a566-a6d7a8339d19 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:14.102295 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-4a61f401-1e7c-469e-a566-a6d7a8339d19 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:14.102793 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-4a61f401-1e7c-469e-a566-a6d7a8339d19 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:14.257065 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-4a61f401-1e7c-469e-a566-a6d7a8339d19 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd3f0d615187d4779bdd7da8c9f24c796', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0151d58abd047a692f5cc4e7d05bea5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:42:14.267063 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-4a61f401-1e7c-469e-a566-a6d7a8339d19 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] 10.222.0.26 "GET /compute/v2.1/servers/3da61834-fd67-49d2-925c-c18fc264042c" status: 200 len: 1521 microversion: 2.1 time: 0.181130 Oct 19 00:42:14.267749 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 93/183] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:42:14 2022] GET /compute/v2.1/servers/3da61834-fd67-49d2-925c-c18fc264042c => generated 1521 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 19 00:42:14.368636 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi [None req-8b1b8117-3685-407a-a94a-5a367d773d94 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Calling method '>' {{(pid=125640) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:42:14.369367 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.compute.api [None req-8b1b8117-3685-407a-a94a-5a367d773d94 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] [instance: 66990043-23b4-4151-8137-6637e4b0a1db] Fetching instance by UUID {{(pid=125640) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:42:14.375733 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-8b1b8117-3685-407a-a94a-5a367d773d94 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:14.376233 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-8b1b8117-3685-407a-a94a-5a367d773d94 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:14.376677 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-8b1b8117-3685-407a-a94a-5a367d773d94 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:14.377391 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-8b1b8117-3685-407a-a94a-5a367d773d94 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:14.377849 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-8b1b8117-3685-407a-a94a-5a367d773d94 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:14.378276 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-8b1b8117-3685-407a-a94a-5a367d773d94 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:14.528104 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.policy [None req-8b1b8117-3685-407a-a94a-5a367d773d94 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a08c9457231a447eb1c4130e68cfb31d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '173a52b79dab4962a4aa380c26f97d2d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125640) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:42:14.536443 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: INFO nova.api.openstack.requestlog [None req-8b1b8117-3685-407a-a94a-5a367d773d94 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] 10.222.0.26 "GET /compute/v2.1/servers/66990043-23b4-4151-8137-6637e4b0a1db" status: 200 len: 1314 microversion: 2.1 time: 0.172123 Oct 19 00:42:14.537009 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: [pid: 125640|app: 0|req: 91/184] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:42:14 2022] GET /compute/v2.1/servers/66990043-23b4-4151-8137-6637e4b0a1db => generated 1314 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:42:15.291353 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-d2d8976c-f43b-4a1c-992b-4dd634d53aad tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:42:15.292046 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-d2d8976c-f43b-4a1c-992b-4dd634d53aad tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] [instance: 3da61834-fd67-49d2-925c-c18fc264042c] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:42:15.298511 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d2d8976c-f43b-4a1c-992b-4dd634d53aad tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:15.298991 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d2d8976c-f43b-4a1c-992b-4dd634d53aad tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:15.299422 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d2d8976c-f43b-4a1c-992b-4dd634d53aad tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:15.300108 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d2d8976c-f43b-4a1c-992b-4dd634d53aad tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:15.300551 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d2d8976c-f43b-4a1c-992b-4dd634d53aad tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:15.300968 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d2d8976c-f43b-4a1c-992b-4dd634d53aad tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:15.459597 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-d2d8976c-f43b-4a1c-992b-4dd634d53aad tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd3f0d615187d4779bdd7da8c9f24c796', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0151d58abd047a692f5cc4e7d05bea5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:42:15.468608 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-d2d8976c-f43b-4a1c-992b-4dd634d53aad tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] 10.222.0.26 "GET /compute/v2.1/servers/3da61834-fd67-49d2-925c-c18fc264042c" status: 200 len: 1521 microversion: 2.1 time: 0.180434 Oct 19 00:42:15.469319 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 94/185] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:42:15 2022] GET /compute/v2.1/servers/3da61834-fd67-49d2-925c-c18fc264042c => generated 1521 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:42:15.562758 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi [None req-86746146-4cfd-42e8-8a65-a3cf688c3109 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Calling method '>' {{(pid=125640) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:42:15.573694 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.compute.api [None req-86746146-4cfd-42e8-8a65-a3cf688c3109 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] [instance: 66990043-23b4-4151-8137-6637e4b0a1db] Fetching instance by UUID {{(pid=125640) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:42:15.582688 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-86746146-4cfd-42e8-8a65-a3cf688c3109 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:15.583270 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-86746146-4cfd-42e8-8a65-a3cf688c3109 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:15.583715 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-86746146-4cfd-42e8-8a65-a3cf688c3109 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:15.584419 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-86746146-4cfd-42e8-8a65-a3cf688c3109 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:15.584878 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-86746146-4cfd-42e8-8a65-a3cf688c3109 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:15.585490 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-86746146-4cfd-42e8-8a65-a3cf688c3109 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:15.726566 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.policy [None req-86746146-4cfd-42e8-8a65-a3cf688c3109 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a08c9457231a447eb1c4130e68cfb31d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '173a52b79dab4962a4aa380c26f97d2d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125640) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:42:15.734416 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: INFO nova.api.openstack.requestlog [None req-86746146-4cfd-42e8-8a65-a3cf688c3109 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] 10.222.0.26 "GET /compute/v2.1/servers/66990043-23b4-4151-8137-6637e4b0a1db" status: 200 len: 1314 microversion: 2.1 time: 0.176815 Oct 19 00:42:15.735089 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: [pid: 125640|app: 0|req: 92/186] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:42:15 2022] GET /compute/v2.1/servers/66990043-23b4-4151-8137-6637e4b0a1db => generated 1314 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:42:16.494415 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-0cd9ed25-acf7-4e22-8847-763a58171109 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:42:16.495323 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-0cd9ed25-acf7-4e22-8847-763a58171109 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] [instance: 3da61834-fd67-49d2-925c-c18fc264042c] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:42:16.502094 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-0cd9ed25-acf7-4e22-8847-763a58171109 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:16.502849 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-0cd9ed25-acf7-4e22-8847-763a58171109 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:16.503285 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-0cd9ed25-acf7-4e22-8847-763a58171109 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:16.503981 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-0cd9ed25-acf7-4e22-8847-763a58171109 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:16.504426 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-0cd9ed25-acf7-4e22-8847-763a58171109 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:16.504894 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-0cd9ed25-acf7-4e22-8847-763a58171109 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:16.643943 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-0cd9ed25-acf7-4e22-8847-763a58171109 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd3f0d615187d4779bdd7da8c9f24c796', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0151d58abd047a692f5cc4e7d05bea5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:42:16.653350 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-0cd9ed25-acf7-4e22-8847-763a58171109 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] 10.222.0.26 "GET /compute/v2.1/servers/3da61834-fd67-49d2-925c-c18fc264042c" status: 200 len: 1521 microversion: 2.1 time: 0.162917 Oct 19 00:42:16.654128 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 95/187] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:42:16 2022] GET /compute/v2.1/servers/3da61834-fd67-49d2-925c-c18fc264042c => generated 1521 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 19 00:42:16.757440 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi [None req-21a9beeb-ace2-4e7d-b5ca-1db696b035ca tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Calling method '>' {{(pid=125640) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:42:16.758717 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.compute.api [None req-21a9beeb-ace2-4e7d-b5ca-1db696b035ca tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] [instance: 66990043-23b4-4151-8137-6637e4b0a1db] Fetching instance by UUID {{(pid=125640) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:42:16.766323 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-21a9beeb-ace2-4e7d-b5ca-1db696b035ca tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:16.766927 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-21a9beeb-ace2-4e7d-b5ca-1db696b035ca tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:16.767365 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-21a9beeb-ace2-4e7d-b5ca-1db696b035ca tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:16.768095 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-21a9beeb-ace2-4e7d-b5ca-1db696b035ca tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:16.768728 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-21a9beeb-ace2-4e7d-b5ca-1db696b035ca tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:16.769159 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-21a9beeb-ace2-4e7d-b5ca-1db696b035ca tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:16.785141 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG dbcounter [-] [125640] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=125640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 00:42:16.923181 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.policy [None req-21a9beeb-ace2-4e7d-b5ca-1db696b035ca tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a08c9457231a447eb1c4130e68cfb31d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '173a52b79dab4962a4aa380c26f97d2d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125640) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:42:16.932470 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: INFO nova.api.openstack.requestlog [None req-21a9beeb-ace2-4e7d-b5ca-1db696b035ca tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] 10.222.0.26 "GET /compute/v2.1/servers/66990043-23b4-4151-8137-6637e4b0a1db" status: 200 len: 1497 microversion: 2.1 time: 0.178961 Oct 19 00:42:16.933052 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: [pid: 125640|app: 0|req: 93/188] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:42:16 2022] GET /compute/v2.1/servers/66990043-23b4-4151-8137-6637e4b0a1db => generated 1497 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:42:17.680240 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-add72b39-cbe2-438b-b754-adf9f57abfc9 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:42:17.680848 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-add72b39-cbe2-438b-b754-adf9f57abfc9 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] [instance: 3da61834-fd67-49d2-925c-c18fc264042c] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:42:17.686848 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-add72b39-cbe2-438b-b754-adf9f57abfc9 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:17.687323 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-add72b39-cbe2-438b-b754-adf9f57abfc9 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:17.687782 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-add72b39-cbe2-438b-b754-adf9f57abfc9 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:17.688460 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-add72b39-cbe2-438b-b754-adf9f57abfc9 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:17.688907 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-add72b39-cbe2-438b-b754-adf9f57abfc9 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:17.689324 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-add72b39-cbe2-438b-b754-adf9f57abfc9 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:17.828806 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-add72b39-cbe2-438b-b754-adf9f57abfc9 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd3f0d615187d4779bdd7da8c9f24c796', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0151d58abd047a692f5cc4e7d05bea5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:42:17.837064 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-add72b39-cbe2-438b-b754-adf9f57abfc9 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] 10.222.0.26 "GET /compute/v2.1/servers/3da61834-fd67-49d2-925c-c18fc264042c" status: 200 len: 1521 microversion: 2.1 time: 0.160529 Oct 19 00:42:17.837642 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 96/189] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:42:17 2022] GET /compute/v2.1/servers/3da61834-fd67-49d2-925c-c18fc264042c => generated 1521 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:42:17.869907 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi [None req-abdce3d8-7748-4b00-aa5f-d902fc7c32ce tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Calling method '>' {{(pid=125640) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:42:17.870902 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.compute.api [None req-abdce3d8-7748-4b00-aa5f-d902fc7c32ce tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] [instance: 66990043-23b4-4151-8137-6637e4b0a1db] Fetching instance by UUID {{(pid=125640) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:42:17.877559 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-abdce3d8-7748-4b00-aa5f-d902fc7c32ce tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:17.878039 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-abdce3d8-7748-4b00-aa5f-d902fc7c32ce tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:17.878473 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-abdce3d8-7748-4b00-aa5f-d902fc7c32ce tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:17.879178 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-abdce3d8-7748-4b00-aa5f-d902fc7c32ce tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:17.879628 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-abdce3d8-7748-4b00-aa5f-d902fc7c32ce tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:17.880049 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-abdce3d8-7748-4b00-aa5f-d902fc7c32ce tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:17.904910 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.compute.api [None req-abdce3d8-7748-4b00-aa5f-d902fc7c32ce tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] [instance: 66990043-23b4-4151-8137-6637e4b0a1db] Going to try to terminate instance {{(pid=125640) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 19 00:42:17.996046 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: INFO nova.api.openstack.requestlog [None req-abdce3d8-7748-4b00-aa5f-d902fc7c32ce tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] 10.222.0.26 "DELETE /compute/v2.1/servers/66990043-23b4-4151-8137-6637e4b0a1db" status: 204 len: 0 microversion: 2.1 time: 0.129678 Oct 19 00:42:17.996688 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: [pid: 125640|app: 0|req: 94/190] 10.222.0.26 () {58 vars in 1239 bytes} [Wed Oct 19 00:42:17 2022] DELETE /compute/v2.1/servers/66990043-23b4-4151-8137-6637e4b0a1db => generated 0 bytes in 131 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 19 00:42:18.005674 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-a968f414-7ac9-41df-9ab8-9024a3ed27b5 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:42:18.006279 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-a968f414-7ac9-41df-9ab8-9024a3ed27b5 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] [instance: 66990043-23b4-4151-8137-6637e4b0a1db] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:42:18.012223 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-a968f414-7ac9-41df-9ab8-9024a3ed27b5 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:18.012695 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-a968f414-7ac9-41df-9ab8-9024a3ed27b5 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:18.013122 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-a968f414-7ac9-41df-9ab8-9024a3ed27b5 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:18.013794 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-a968f414-7ac9-41df-9ab8-9024a3ed27b5 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:18.014239 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-a968f414-7ac9-41df-9ab8-9024a3ed27b5 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:18.015456 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-a968f414-7ac9-41df-9ab8-9024a3ed27b5 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:18.035261 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG dbcounter [-] [125641] Writing DB stats nova_cell1:SELECT=100 {{(pid=125641) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 00:42:18.152887 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-a968f414-7ac9-41df-9ab8-9024a3ed27b5 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a08c9457231a447eb1c4130e68cfb31d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '173a52b79dab4962a4aa380c26f97d2d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:42:18.167937 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-a968f414-7ac9-41df-9ab8-9024a3ed27b5 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] 10.222.0.26 "GET /compute/v2.1/servers/66990043-23b4-4151-8137-6637e4b0a1db" status: 200 len: 1503 microversion: 2.1 time: 0.159651 Oct 19 00:42:18.167937 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 97/191] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:42:18 2022] GET /compute/v2.1/servers/66990043-23b4-4151-8137-6637e4b0a1db => generated 1503 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 19 00:42:18.862104 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi [None req-f52926ec-cfe7-4678-b0ef-dd9555b75784 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Calling method '>' {{(pid=125640) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:42:18.862770 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.compute.api [None req-f52926ec-cfe7-4678-b0ef-dd9555b75784 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] [instance: 3da61834-fd67-49d2-925c-c18fc264042c] Fetching instance by UUID {{(pid=125640) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:42:18.869277 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-f52926ec-cfe7-4678-b0ef-dd9555b75784 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:18.869780 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-f52926ec-cfe7-4678-b0ef-dd9555b75784 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:18.870371 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-f52926ec-cfe7-4678-b0ef-dd9555b75784 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:18.871266 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-f52926ec-cfe7-4678-b0ef-dd9555b75784 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:18.871728 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-f52926ec-cfe7-4678-b0ef-dd9555b75784 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:18.872262 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-f52926ec-cfe7-4678-b0ef-dd9555b75784 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:19.011142 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.policy [None req-f52926ec-cfe7-4678-b0ef-dd9555b75784 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd3f0d615187d4779bdd7da8c9f24c796', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0151d58abd047a692f5cc4e7d05bea5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125640) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:42:19.019647 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: INFO nova.api.openstack.requestlog [None req-f52926ec-cfe7-4678-b0ef-dd9555b75784 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] 10.222.0.26 "GET /compute/v2.1/servers/3da61834-fd67-49d2-925c-c18fc264042c" status: 200 len: 1521 microversion: 2.1 time: 0.162670 Oct 19 00:42:19.020206 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: [pid: 125640|app: 0|req: 95/192] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:42:18 2022] GET /compute/v2.1/servers/3da61834-fd67-49d2-925c-c18fc264042c => generated 1521 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:42:19.185240 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-1caf8201-2724-4ddf-b0e1-91be48450d1b tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:42:19.185856 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-1caf8201-2724-4ddf-b0e1-91be48450d1b tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] [instance: 66990043-23b4-4151-8137-6637e4b0a1db] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:42:19.192397 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-1caf8201-2724-4ddf-b0e1-91be48450d1b tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:19.192868 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-1caf8201-2724-4ddf-b0e1-91be48450d1b tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:19.193294 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-1caf8201-2724-4ddf-b0e1-91be48450d1b tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:19.193968 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-1caf8201-2724-4ddf-b0e1-91be48450d1b tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:19.194415 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-1caf8201-2724-4ddf-b0e1-91be48450d1b tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:19.194863 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-1caf8201-2724-4ddf-b0e1-91be48450d1b tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:19.333818 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-1caf8201-2724-4ddf-b0e1-91be48450d1b tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a08c9457231a447eb1c4130e68cfb31d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '173a52b79dab4962a4aa380c26f97d2d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:42:19.341628 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-1caf8201-2724-4ddf-b0e1-91be48450d1b tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] 10.222.0.26 "GET /compute/v2.1/servers/66990043-23b4-4151-8137-6637e4b0a1db" status: 200 len: 1503 microversion: 2.1 time: 0.160240 Oct 19 00:42:19.342284 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 98/193] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:42:19 2022] GET /compute/v2.1/servers/66990043-23b4-4151-8137-6637e4b0a1db => generated 1503 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:42:20.044332 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi [None req-ddd8733e-fd76-4e6c-ba70-00a602e66bdd tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Calling method '>' {{(pid=125640) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:42:20.045590 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.compute.api [None req-ddd8733e-fd76-4e6c-ba70-00a602e66bdd tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] [instance: 3da61834-fd67-49d2-925c-c18fc264042c] Fetching instance by UUID {{(pid=125640) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:42:20.052523 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-ddd8733e-fd76-4e6c-ba70-00a602e66bdd tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:20.053019 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-ddd8733e-fd76-4e6c-ba70-00a602e66bdd tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:20.053462 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-ddd8733e-fd76-4e6c-ba70-00a602e66bdd tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:20.054312 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-ddd8733e-fd76-4e6c-ba70-00a602e66bdd tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:20.054874 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-ddd8733e-fd76-4e6c-ba70-00a602e66bdd tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:20.055230 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-ddd8733e-fd76-4e6c-ba70-00a602e66bdd tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:20.303118 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.policy [None req-ddd8733e-fd76-4e6c-ba70-00a602e66bdd tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd3f0d615187d4779bdd7da8c9f24c796', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0151d58abd047a692f5cc4e7d05bea5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125640) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:42:20.319393 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: INFO nova.api.openstack.requestlog [None req-ddd8733e-fd76-4e6c-ba70-00a602e66bdd tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] 10.222.0.26 "GET /compute/v2.1/servers/3da61834-fd67-49d2-925c-c18fc264042c" status: 200 len: 1521 microversion: 2.1 time: 0.278709 Oct 19 00:42:20.321191 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: [pid: 125640|app: 0|req: 96/194] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:42:20 2022] GET /compute/v2.1/servers/3da61834-fd67-49d2-925c-c18fc264042c => generated 1521 bytes in 281 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:42:20.366384 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-6eba9045-b83a-4624-91ad-d0e9c107ddb2 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:42:20.367296 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-6eba9045-b83a-4624-91ad-d0e9c107ddb2 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] [instance: 66990043-23b4-4151-8137-6637e4b0a1db] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:42:20.373724 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-6eba9045-b83a-4624-91ad-d0e9c107ddb2 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:20.374864 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-6eba9045-b83a-4624-91ad-d0e9c107ddb2 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:20.375496 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-6eba9045-b83a-4624-91ad-d0e9c107ddb2 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:20.376118 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-6eba9045-b83a-4624-91ad-d0e9c107ddb2 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:20.376648 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-6eba9045-b83a-4624-91ad-d0e9c107ddb2 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:20.377180 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-6eba9045-b83a-4624-91ad-d0e9c107ddb2 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:20.549850 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-6eba9045-b83a-4624-91ad-d0e9c107ddb2 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a08c9457231a447eb1c4130e68cfb31d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '173a52b79dab4962a4aa380c26f97d2d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:42:20.559929 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-6eba9045-b83a-4624-91ad-d0e9c107ddb2 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] 10.222.0.26 "GET /compute/v2.1/servers/66990043-23b4-4151-8137-6637e4b0a1db" status: 200 len: 1503 microversion: 2.1 time: 0.197310 Oct 19 00:42:20.560667 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 99/195] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:42:20 2022] GET /compute/v2.1/servers/66990043-23b4-4151-8137-6637e4b0a1db => generated 1503 bytes in 199 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 19 00:42:20.642288 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi [req-67ef9cfa-421a-48ff-85d6-d19225e0fa24 req-e739f76c-378d-4a43-b1d2-3784cfc17e51 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "66990043-23b4-4151-8137-6637e4b0a1db", "name": "network-vif-deleted", "tag": "4d3e0aaa-9ce0-44ac-86e1-3ac2ffbc0ec1"}]} {{(pid=125640) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 19 00:42:20.654807 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [req-67ef9cfa-421a-48ff-85d6-d19225e0fa24 req-e739f76c-378d-4a43-b1d2-3784cfc17e51 service nova] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:20.655410 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [req-67ef9cfa-421a-48ff-85d6-d19225e0fa24 req-e739f76c-378d-4a43-b1d2-3784cfc17e51 service nova] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:20.655914 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [req-67ef9cfa-421a-48ff-85d6-d19225e0fa24 req-e739f76c-378d-4a43-b1d2-3784cfc17e51 service nova] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:20.667704 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: INFO nova.api.openstack.compute.server_external_events [req-67ef9cfa-421a-48ff-85d6-d19225e0fa24 req-e739f76c-378d-4a43-b1d2-3784cfc17e51 service nova] Creating event network-vif-deleted:4d3e0aaa-9ce0-44ac-86e1-3ac2ffbc0ec1 for instance 66990043-23b4-4151-8137-6637e4b0a1db on hv-cinder-maste Oct 19 00:42:20.673883 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: INFO nova.api.openstack.requestlog [req-67ef9cfa-421a-48ff-85d6-d19225e0fa24 req-e739f76c-378d-4a43-b1d2-3784cfc17e51 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.035728 Oct 19 00:42:20.674421 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: [pid: 125640|app: 0|req: 97/196] 10.222.0.26 () {64 vars in 1317 bytes} [Wed Oct 19 00:42:20 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 19 00:42:21.347844 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-d2ebfc74-c80f-454b-9421-be17ad4220c3 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:42:21.348588 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-d2ebfc74-c80f-454b-9421-be17ad4220c3 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] [instance: 3da61834-fd67-49d2-925c-c18fc264042c] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:42:21.357136 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d2ebfc74-c80f-454b-9421-be17ad4220c3 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:21.357638 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d2ebfc74-c80f-454b-9421-be17ad4220c3 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:21.358111 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d2ebfc74-c80f-454b-9421-be17ad4220c3 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:21.359020 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d2ebfc74-c80f-454b-9421-be17ad4220c3 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:21.359488 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d2ebfc74-c80f-454b-9421-be17ad4220c3 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:21.359918 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d2ebfc74-c80f-454b-9421-be17ad4220c3 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:21.507831 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-d2ebfc74-c80f-454b-9421-be17ad4220c3 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd3f0d615187d4779bdd7da8c9f24c796', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0151d58abd047a692f5cc4e7d05bea5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:42:21.517761 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-d2ebfc74-c80f-454b-9421-be17ad4220c3 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] 10.222.0.26 "GET /compute/v2.1/servers/3da61834-fd67-49d2-925c-c18fc264042c" status: 200 len: 1521 microversion: 2.1 time: 0.174956 Oct 19 00:42:21.518706 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 100/197] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:42:21 2022] GET /compute/v2.1/servers/3da61834-fd67-49d2-925c-c18fc264042c => generated 1521 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 19 00:42:21.589734 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi [None req-082bc609-185d-497e-ae71-2367abe357eb tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Calling method '>' {{(pid=125640) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:42:21.590424 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.compute.api [None req-082bc609-185d-497e-ae71-2367abe357eb tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] [instance: 66990043-23b4-4151-8137-6637e4b0a1db] Fetching instance by UUID {{(pid=125640) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:42:21.597170 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-082bc609-185d-497e-ae71-2367abe357eb tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:21.597650 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-082bc609-185d-497e-ae71-2367abe357eb tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:21.598077 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-082bc609-185d-497e-ae71-2367abe357eb tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:21.598785 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-082bc609-185d-497e-ae71-2367abe357eb tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:21.599235 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-082bc609-185d-497e-ae71-2367abe357eb tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:21.599650 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-082bc609-185d-497e-ae71-2367abe357eb tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:21.613873 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: INFO nova.api.openstack.wsgi [None req-082bc609-185d-497e-ae71-2367abe357eb tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] HTTP exception thrown: Instance 66990043-23b4-4151-8137-6637e4b0a1db could not be found. Oct 19 00:42:21.616013 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi [None req-082bc609-185d-497e-ae71-2367abe357eb tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Request method failure captured: Oct 19 00:42:21.616013 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: request: GET /compute/v2.1/servers/66990043-23b4-4151-8137-6637e4b0a1db HTTP/1.1 Oct 19 00:42:21.616013 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: Accept: application/json Oct 19 00:42:21.616013 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: Accept-Encoding: identity Oct 19 00:42:21.616013 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: Connection: close Oct 19 00:42:21.616013 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: Content-Length: 0 Oct 19 00:42:21.616013 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: Content-Type: application/json Oct 19 00:42:21.616013 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: Host: 10.222.0.26 Oct 19 00:42:21.616013 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: Openstack-System-Scope: None Oct 19 00:42:21.616013 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: User-Agent: python-urllib3/1.26.12 Oct 19 00:42:21.616013 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: X-Auth-Token: gAAAAABjT0fWq8PufKfTNVGCB8UZN8rquspHB0M18f9BpiVGpFdRUCE9-sef668qbt4mHsvxBbayoK7Q4ESz7O8Fc51IujD1f1lyiIXi0P75C3OdhQ8yI0wWhqAReMDCiWL9XbVVxqVdF3iwbCfkbNEiXK-RJrPPtPv4_Nq7EBUKZW6gSE2wfW8 Oct 19 00:42:21.616013 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: X-Domain-Id: None Oct 19 00:42:21.616013 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: X-Domain-Name: None Oct 19 00:42:21.616013 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: X-Identity-Status: Confirmed Oct 19 00:42:21.616013 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: X-Is-Admin-Project: True Oct 19 00:42:21.616013 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: X-Project-Domain-Id: default Oct 19 00:42:21.616013 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: X-Project-Domain-Name: Default Oct 19 00:42:21.616013 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: X-Project-Id: 173a52b79dab4962a4aa380c26f97d2d Oct 19 00:42:21.616013 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: X-Project-Name: tempest-VolumesActionsTest-2037902882 Oct 19 00:42:21.616013 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: X-Role: member,reader Oct 19 00:42:21.616013 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: X-Roles: member,reader Oct 19 00:42:21.617712 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/173a52b79dab4962a4aa380c26f97d2d"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/173a52b79dab4962a4aa380c26f97d2d"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/173a52b79dab4962a4aa380c26f97d2d"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_173a52b79dab4962a4aa380c26f97d2d"}]}] Oct 19 00:42:21.617712 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: X-Tenant: tempest-VolumesActionsTest-2037902882 Oct 19 00:42:21.617712 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: X-Tenant-Id: 173a52b79dab4962a4aa380c26f97d2d Oct 19 00:42:21.617712 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: X-Tenant-Name: tempest-VolumesActionsTest-2037902882 Oct 19 00:42:21.617712 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: X-User: tempest-VolumesActionsTest-2037902882-project Oct 19 00:42:21.617712 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: X-User-Domain-Id: default Oct 19 00:42:21.617712 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: X-User-Domain-Name: Default Oct 19 00:42:21.617712 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: X-User-Id: a08c9457231a447eb1c4130e68cfb31d Oct 19 00:42:21.617712 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: X-User-Name: tempest-VolumesActionsTest-2037902882-project Oct 19 00:42:21.617712 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: method: > Oct 19 00:42:21.617712 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: exception: Instance 66990043-23b4-4151-8137-6637e4b0a1db could not be found. Oct 19 00:42:21.617712 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: action_args: {'id': '66990043-23b4-4151-8137-6637e4b0a1db'} Oct 19 00:42:21.631028 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: {{(pid=125640) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 19 00:42:21.631028 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 19 00:42:21.631028 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 19 00:42:21.631028 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 19 00:42:21.631028 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 19 00:42:21.631028 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 19 00:42:21.631028 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 19 00:42:21.631028 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 19 00:42:21.631028 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 19 00:42:21.631028 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 19 00:42:21.631028 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 66990043-23b4-4151-8137-6637e4b0a1db could not be found. Oct 19 00:42:21.631028 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi Oct 19 00:42:21.631028 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 19 00:42:21.631028 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi Oct 19 00:42:21.631028 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 19 00:42:21.631028 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 19 00:42:21.631028 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 19 00:42:21.631028 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 19 00:42:21.631028 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 19 00:42:21.631028 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 19 00:42:21.632399 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 19 00:42:21.632399 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 19 00:42:21.632399 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 19 00:42:21.632399 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 19 00:42:21.632399 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 19 00:42:21.632399 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 19 00:42:21.632399 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 19 00:42:21.632399 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 66990043-23b4-4151-8137-6637e4b0a1db could not be found. Oct 19 00:42:21.632399 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi Oct 19 00:42:21.632399 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 19 00:42:21.632399 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi Oct 19 00:42:21.632399 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 19 00:42:21.632399 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 19 00:42:21.632399 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 19 00:42:21.632399 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 19 00:42:21.632399 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 19 00:42:21.632399 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 66990043-23b4-4151-8137-6637e4b0a1db could not be found. Oct 19 00:42:21.632399 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: ERROR nova.api.openstack.wsgi Oct 19 00:42:21.633560 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi [None req-082bc609-185d-497e-ae71-2367abe357eb tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Returning 404 to user: Instance 66990043-23b4-4151-8137-6637e4b0a1db could not be found. {{(pid=125640) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 19 00:42:21.633560 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: INFO nova.api.openstack.requestlog [None req-082bc609-185d-497e-ae71-2367abe357eb tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] 10.222.0.26 "GET /compute/v2.1/servers/66990043-23b4-4151-8137-6637e4b0a1db" status: 404 len: 111 microversion: 2.1 time: 0.032445 Oct 19 00:42:21.633560 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: [pid: 125640|app: 0|req: 98/198] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:42:21 2022] GET /compute/v2.1/servers/66990043-23b4-4151-8137-6637e4b0a1db => generated 111 bytes in 34 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 19 00:42:21.745237 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-df4245d4-3596-4e5f-b919-776cb2de4e3e tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesActionsTest-instance-1027679876", "imageRef": "b387b4f8-f75e-4a68-b49c-a015bf61c46b", "flavorRef": "452", "networks": [{"uuid": "7d7558f7-ec39-4fba-83f7-3ca748451727"}]}} {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 19 00:42:22.542080 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi [None req-2a11041d-1666-4626-9b78-6bef5aebc3fb tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Calling method '>' {{(pid=125640) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:42:22.542725 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.compute.api [None req-2a11041d-1666-4626-9b78-6bef5aebc3fb tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] [instance: 3da61834-fd67-49d2-925c-c18fc264042c] Fetching instance by UUID {{(pid=125640) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:42:22.550739 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-2a11041d-1666-4626-9b78-6bef5aebc3fb tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:22.551485 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-2a11041d-1666-4626-9b78-6bef5aebc3fb tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:22.551929 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-2a11041d-1666-4626-9b78-6bef5aebc3fb tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:22.552653 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-2a11041d-1666-4626-9b78-6bef5aebc3fb tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:22.553123 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-2a11041d-1666-4626-9b78-6bef5aebc3fb tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:22.553583 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-2a11041d-1666-4626-9b78-6bef5aebc3fb tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:22.696896 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.policy [None req-2a11041d-1666-4626-9b78-6bef5aebc3fb tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd3f0d615187d4779bdd7da8c9f24c796', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0151d58abd047a692f5cc4e7d05bea5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125640) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:42:22.705389 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: INFO nova.api.openstack.requestlog [None req-2a11041d-1666-4626-9b78-6bef5aebc3fb tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] 10.222.0.26 "GET /compute/v2.1/servers/3da61834-fd67-49d2-925c-c18fc264042c" status: 200 len: 1521 microversion: 2.1 time: 0.167058 Oct 19 00:42:22.705956 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: [pid: 125640|app: 0|req: 99/199] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:42:22 2022] GET /compute/v2.1/servers/3da61834-fd67-49d2-925c-c18fc264042c => generated 1521 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 19 00:42:23.729252 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi [None req-55531151-e3fb-470b-b2b2-b8b216bdf6ab tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Calling method '>' {{(pid=125640) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:42:23.729866 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.compute.api [None req-55531151-e3fb-470b-b2b2-b8b216bdf6ab tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] [instance: 3da61834-fd67-49d2-925c-c18fc264042c] Fetching instance by UUID {{(pid=125640) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:42:23.737386 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-55531151-e3fb-470b-b2b2-b8b216bdf6ab tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:23.737905 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-55531151-e3fb-470b-b2b2-b8b216bdf6ab tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:23.738342 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-55531151-e3fb-470b-b2b2-b8b216bdf6ab tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:23.739077 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-55531151-e3fb-470b-b2b2-b8b216bdf6ab tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:23.739534 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-55531151-e3fb-470b-b2b2-b8b216bdf6ab tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:23.739962 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-55531151-e3fb-470b-b2b2-b8b216bdf6ab tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:24.012624 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.policy [None req-55531151-e3fb-470b-b2b2-b8b216bdf6ab tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd3f0d615187d4779bdd7da8c9f24c796', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0151d58abd047a692f5cc4e7d05bea5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125640) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:42:24.020908 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: INFO nova.api.openstack.requestlog [None req-55531151-e3fb-470b-b2b2-b8b216bdf6ab tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] 10.222.0.26 "GET /compute/v2.1/servers/3da61834-fd67-49d2-925c-c18fc264042c" status: 200 len: 1521 microversion: 2.1 time: 0.295567 Oct 19 00:42:24.021463 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: [pid: 125640|app: 0|req: 100/200] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:42:23 2022] GET /compute/v2.1/servers/3da61834-fd67-49d2-925c-c18fc264042c => generated 1521 bytes in 297 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 19 00:42:24.025914 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi [None req-4a58984b-4aab-4188-938c-9ac5a14005da tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Calling method '>' {{(pid=125640) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:42:24.026396 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.compute.api [None req-4a58984b-4aab-4188-938c-9ac5a14005da tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] [instance: dc8b8a6e-eab7-4f45-8f5d-916c52eec398] Fetching instance by UUID {{(pid=125640) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:42:24.033057 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-4a58984b-4aab-4188-938c-9ac5a14005da tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:24.033534 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-4a58984b-4aab-4188-938c-9ac5a14005da tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:24.033964 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-4a58984b-4aab-4188-938c-9ac5a14005da tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:24.034708 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-4a58984b-4aab-4188-938c-9ac5a14005da tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:24.035247 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-4a58984b-4aab-4188-938c-9ac5a14005da tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:24.035857 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-4a58984b-4aab-4188-938c-9ac5a14005da tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:24.060369 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: INFO nova.api.openstack.requestlog [None req-4a58984b-4aab-4188-938c-9ac5a14005da tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] 10.222.0.26 "GET /compute/v2.1/servers/dc8b8a6e-eab7-4f45-8f5d-916c52eec398/ips" status: 200 len: 147 microversion: 2.1 time: 0.037859 Oct 19 00:42:24.060937 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: [pid: 125640|app: 0|req: 101/201] 10.222.0.26 () {58 vars in 1248 bytes} [Wed Oct 19 00:42:24 2022] GET /compute/v2.1/servers/dc8b8a6e-eab7-4f45-8f5d-916c52eec398/ips => generated 147 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 19 00:42:25.044940 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi [None req-11f7f00c-fcf6-41d6-9778-a669ed695b9c tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Calling method '>' {{(pid=125640) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:42:25.046134 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.compute.api [None req-11f7f00c-fcf6-41d6-9778-a669ed695b9c tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] [instance: 3da61834-fd67-49d2-925c-c18fc264042c] Fetching instance by UUID {{(pid=125640) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:42:25.052828 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-11f7f00c-fcf6-41d6-9778-a669ed695b9c tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:25.053312 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-11f7f00c-fcf6-41d6-9778-a669ed695b9c tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:25.053748 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-11f7f00c-fcf6-41d6-9778-a669ed695b9c tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:25.054444 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-11f7f00c-fcf6-41d6-9778-a669ed695b9c tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:25.054929 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-11f7f00c-fcf6-41d6-9778-a669ed695b9c tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:25.055374 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-11f7f00c-fcf6-41d6-9778-a669ed695b9c tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:25.190449 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.policy [None req-11f7f00c-fcf6-41d6-9778-a669ed695b9c tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd3f0d615187d4779bdd7da8c9f24c796', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0151d58abd047a692f5cc4e7d05bea5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125640) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:42:25.198132 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: INFO nova.api.openstack.requestlog [None req-11f7f00c-fcf6-41d6-9778-a669ed695b9c tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] 10.222.0.26 "GET /compute/v2.1/servers/3da61834-fd67-49d2-925c-c18fc264042c" status: 200 len: 1521 microversion: 2.1 time: 0.157070 Oct 19 00:42:25.198747 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: [pid: 125640|app: 0|req: 102/202] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:42:25 2022] GET /compute/v2.1/servers/3da61834-fd67-49d2-925c-c18fc264042c => generated 1521 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 19 00:42:26.221544 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi [None req-c333f7bf-bd76-4dc0-9d14-6b1b9e9f52f3 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Calling method '>' {{(pid=125640) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:42:26.222218 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.compute.api [None req-c333f7bf-bd76-4dc0-9d14-6b1b9e9f52f3 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] [instance: 3da61834-fd67-49d2-925c-c18fc264042c] Fetching instance by UUID {{(pid=125640) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:42:26.228584 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-c333f7bf-bd76-4dc0-9d14-6b1b9e9f52f3 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:26.229064 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-c333f7bf-bd76-4dc0-9d14-6b1b9e9f52f3 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:26.229495 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-c333f7bf-bd76-4dc0-9d14-6b1b9e9f52f3 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:26.230178 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-c333f7bf-bd76-4dc0-9d14-6b1b9e9f52f3 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:26.230668 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-c333f7bf-bd76-4dc0-9d14-6b1b9e9f52f3 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:26.231088 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-c333f7bf-bd76-4dc0-9d14-6b1b9e9f52f3 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:26.364044 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.policy [None req-c333f7bf-bd76-4dc0-9d14-6b1b9e9f52f3 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd3f0d615187d4779bdd7da8c9f24c796', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0151d58abd047a692f5cc4e7d05bea5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125640) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:42:26.372455 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: INFO nova.api.openstack.requestlog [None req-c333f7bf-bd76-4dc0-9d14-6b1b9e9f52f3 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] 10.222.0.26 "GET /compute/v2.1/servers/3da61834-fd67-49d2-925c-c18fc264042c" status: 200 len: 1521 microversion: 2.1 time: 0.155047 Oct 19 00:42:26.373014 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: [pid: 125640|app: 0|req: 103/203] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:42:26 2022] GET /compute/v2.1/servers/3da61834-fd67-49d2-925c-c18fc264042c => generated 1521 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 19 00:42:27.396525 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi [None req-a610083b-4813-4975-b56a-d77ef3bab336 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Calling method '>' {{(pid=125640) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:42:27.397190 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.compute.api [None req-a610083b-4813-4975-b56a-d77ef3bab336 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] [instance: 3da61834-fd67-49d2-925c-c18fc264042c] Fetching instance by UUID {{(pid=125640) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:42:27.404279 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-a610083b-4813-4975-b56a-d77ef3bab336 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:27.404773 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-a610083b-4813-4975-b56a-d77ef3bab336 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:27.405852 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-a610083b-4813-4975-b56a-d77ef3bab336 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:27.406647 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-a610083b-4813-4975-b56a-d77ef3bab336 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:27.407110 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-a610083b-4813-4975-b56a-d77ef3bab336 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:27.407543 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-a610083b-4813-4975-b56a-d77ef3bab336 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:27.541817 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.policy [None req-a610083b-4813-4975-b56a-d77ef3bab336 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd3f0d615187d4779bdd7da8c9f24c796', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0151d58abd047a692f5cc4e7d05bea5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125640) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:42:27.550686 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: INFO nova.api.openstack.requestlog [None req-a610083b-4813-4975-b56a-d77ef3bab336 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] 10.222.0.26 "GET /compute/v2.1/servers/3da61834-fd67-49d2-925c-c18fc264042c" status: 200 len: 1521 microversion: 2.1 time: 0.157950 Oct 19 00:42:27.554404 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: [pid: 125640|app: 0|req: 104/204] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:42:27 2022] GET /compute/v2.1/servers/3da61834-fd67-49d2-925c-c18fc264042c => generated 1521 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 19 00:42:27.558757 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi [None req-90d56ebd-e07e-41c7-bfc3-a7b7846bf42a tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "6ccd7014-9130-42d2-add5-0b6bc74def3f"}} {{(pid=125640) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 19 00:42:27.561086 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.compute.api [None req-90d56ebd-e07e-41c7-bfc3-a7b7846bf42a tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] [instance: dc8b8a6e-eab7-4f45-8f5d-916c52eec398] Fetching instance by UUID {{(pid=125640) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:42:27.567704 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-90d56ebd-e07e-41c7-bfc3-a7b7846bf42a tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:27.568191 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-90d56ebd-e07e-41c7-bfc3-a7b7846bf42a tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:27.568633 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-90d56ebd-e07e-41c7-bfc3-a7b7846bf42a tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:27.569331 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-90d56ebd-e07e-41c7-bfc3-a7b7846bf42a tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:27.569932 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-90d56ebd-e07e-41c7-bfc3-a7b7846bf42a tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:27.570427 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-90d56ebd-e07e-41c7-bfc3-a7b7846bf42a tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:30.271307 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: INFO nova.api.openstack.requestlog [None req-90d56ebd-e07e-41c7-bfc3-a7b7846bf42a tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] 10.222.0.26 "POST /compute/v2.1/servers/dc8b8a6e-eab7-4f45-8f5d-916c52eec398/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 2.716179 Oct 19 00:42:30.278232 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: [pid: 125640|app: 0|req: 105/205] 10.222.0.26 () {60 vars in 1323 bytes} [Wed Oct 19 00:42:27 2022] POST /compute/v2.1/servers/dc8b8a6e-eab7-4f45-8f5d-916c52eec398/os-volume_attachments => generated 194 bytes in 2718 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 19 00:42:30.278232 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi [None req-fd3d6e34-5711-4f11-9593-83c6074bcc12 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Calling method '>' {{(pid=125640) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:42:30.278440 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.compute.api [None req-fd3d6e34-5711-4f11-9593-83c6074bcc12 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] [instance: 3da61834-fd67-49d2-925c-c18fc264042c] Fetching instance by UUID {{(pid=125640) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:42:30.286430 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-fd3d6e34-5711-4f11-9593-83c6074bcc12 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:30.286987 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-fd3d6e34-5711-4f11-9593-83c6074bcc12 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:30.287510 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-fd3d6e34-5711-4f11-9593-83c6074bcc12 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:30.288228 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-fd3d6e34-5711-4f11-9593-83c6074bcc12 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:30.288748 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-fd3d6e34-5711-4f11-9593-83c6074bcc12 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:30.289177 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-fd3d6e34-5711-4f11-9593-83c6074bcc12 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:30.494470 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.policy [None req-fd3d6e34-5711-4f11-9593-83c6074bcc12 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd3f0d615187d4779bdd7da8c9f24c796', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0151d58abd047a692f5cc4e7d05bea5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125640) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:42:30.508071 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: INFO nova.api.openstack.requestlog [None req-fd3d6e34-5711-4f11-9593-83c6074bcc12 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] 10.222.0.26 "GET /compute/v2.1/servers/3da61834-fd67-49d2-925c-c18fc264042c" status: 200 len: 1521 microversion: 2.1 time: 0.234422 Oct 19 00:42:30.508647 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: [pid: 125640|app: 0|req: 106/206] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:42:30 2022] GET /compute/v2.1/servers/3da61834-fd67-49d2-925c-c18fc264042c => generated 1521 bytes in 236 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:42:30.783252 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi [None req-5190c91c-24ea-4200-9f86-ca49d624a548 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Calling method '>' {{(pid=125640) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:42:30.783827 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.compute.api [None req-5190c91c-24ea-4200-9f86-ca49d624a548 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] [instance: b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2] Fetching instance by UUID {{(pid=125640) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:42:30.790494 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-5190c91c-24ea-4200-9f86-ca49d624a548 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:30.791037 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-5190c91c-24ea-4200-9f86-ca49d624a548 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:30.791456 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-5190c91c-24ea-4200-9f86-ca49d624a548 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:30.792165 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-5190c91c-24ea-4200-9f86-ca49d624a548 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:30.792622 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-5190c91c-24ea-4200-9f86-ca49d624a548 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:30.793050 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-5190c91c-24ea-4200-9f86-ca49d624a548 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:30.828828 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: INFO nova.api.openstack.requestlog [None req-5190c91c-24ea-4200-9f86-ca49d624a548 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] 10.222.0.26 "GET /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2/ips" status: 200 len: 151 microversion: 2.20 time: 0.050100 Oct 19 00:42:30.829348 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: [pid: 125640|app: 0|req: 107/207] 10.222.0.26 () {60 vars in 1289 bytes} [Wed Oct 19 00:42:30 2022] GET /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2/ips => generated 151 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Oct 19 00:42:31.513020 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG dbcounter [-] [125641] Writing DB stats nova_cell1:SELECT=20 {{(pid=125641) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 00:42:31.534675 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi [None req-7bbf1208-f862-47b3-9ca5-9faf7faffa9e tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Calling method '>' {{(pid=125640) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:42:31.535294 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.compute.api [None req-7bbf1208-f862-47b3-9ca5-9faf7faffa9e tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] [instance: 3da61834-fd67-49d2-925c-c18fc264042c] Fetching instance by UUID {{(pid=125640) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:42:31.543519 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-7bbf1208-f862-47b3-9ca5-9faf7faffa9e tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:31.544012 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-7bbf1208-f862-47b3-9ca5-9faf7faffa9e tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:31.546649 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-7bbf1208-f862-47b3-9ca5-9faf7faffa9e tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:31.546649 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-7bbf1208-f862-47b3-9ca5-9faf7faffa9e tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:31.546649 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-7bbf1208-f862-47b3-9ca5-9faf7faffa9e tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:31.546649 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-7bbf1208-f862-47b3-9ca5-9faf7faffa9e tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:31.683245 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.policy [None req-7bbf1208-f862-47b3-9ca5-9faf7faffa9e tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd3f0d615187d4779bdd7da8c9f24c796', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0151d58abd047a692f5cc4e7d05bea5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125640) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:42:31.694070 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: INFO nova.api.openstack.requestlog [None req-7bbf1208-f862-47b3-9ca5-9faf7faffa9e tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] 10.222.0.26 "GET /compute/v2.1/servers/3da61834-fd67-49d2-925c-c18fc264042c" status: 200 len: 1521 microversion: 2.1 time: 0.163165 Oct 19 00:42:31.694684 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: [pid: 125640|app: 0|req: 108/208] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:42:31 2022] GET /compute/v2.1/servers/3da61834-fd67-49d2-925c-c18fc264042c => generated 1521 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 19 00:42:31.755984 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG dbcounter [-] [125641] Writing DB stats nova_api:SELECT=31 {{(pid=125641) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 00:42:32.671289 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.network.neutron [None req-df4245d4-3596-4e5f-b919-776cb2de4e3e tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] validate_networks() for [('7d7558f7-ec39-4fba-83f7-3ca748451727', None, None, None, None, None)] {{(pid=125641) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 19 00:42:32.717084 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi [None req-dead2724-dc1c-442c-986b-8581d36f0fa0 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Calling method '>' {{(pid=125640) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:42:32.718892 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.compute.api [None req-dead2724-dc1c-442c-986b-8581d36f0fa0 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] [instance: 3da61834-fd67-49d2-925c-c18fc264042c] Fetching instance by UUID {{(pid=125640) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:42:32.725635 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-dead2724-dc1c-442c-986b-8581d36f0fa0 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:32.726122 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-dead2724-dc1c-442c-986b-8581d36f0fa0 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:32.726596 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-dead2724-dc1c-442c-986b-8581d36f0fa0 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:32.727300 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-dead2724-dc1c-442c-986b-8581d36f0fa0 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:32.727760 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-dead2724-dc1c-442c-986b-8581d36f0fa0 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:32.728185 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-dead2724-dc1c-442c-986b-8581d36f0fa0 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:32.731197 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG dbcounter [-] [125640] Writing DB stats nova_cell1:SELECT=95,nova_cell1:UPDATE=3,nova_cell1:INSERT=2 {{(pid=125640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 00:42:32.886899 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.policy [None req-dead2724-dc1c-442c-986b-8581d36f0fa0 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd3f0d615187d4779bdd7da8c9f24c796', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0151d58abd047a692f5cc4e7d05bea5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125640) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:42:32.896095 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: INFO nova.api.openstack.requestlog [None req-dead2724-dc1c-442c-986b-8581d36f0fa0 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] 10.222.0.26 "GET /compute/v2.1/servers/3da61834-fd67-49d2-925c-c18fc264042c" status: 200 len: 1521 microversion: 2.1 time: 0.182739 Oct 19 00:42:32.896897 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: [pid: 125640|app: 0|req: 109/209] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:42:32 2022] GET /compute/v2.1/servers/3da61834-fd67-49d2-925c-c18fc264042c => generated 1521 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 19 00:42:33.281769 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.virt.hardware [None req-df4245d4-3596-4e5f-b919-776cb2de4e3e tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Flavor limits 0:0:0 {{(pid=125641) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 19 00:42:33.282195 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.virt.hardware [None req-df4245d4-3596-4e5f-b919-776cb2de4e3e tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Image limits 0:0:0 {{(pid=125641) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 19 00:42:33.282734 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.virt.hardware [None req-df4245d4-3596-4e5f-b919-776cb2de4e3e tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Flavor pref 0:0:0 {{(pid=125641) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 19 00:42:33.283148 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.virt.hardware [None req-df4245d4-3596-4e5f-b919-776cb2de4e3e tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Image pref 0:0:0 {{(pid=125641) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 19 00:42:33.283589 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.virt.hardware [None req-df4245d4-3596-4e5f-b919-776cb2de4e3e tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=125641) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 19 00:42:33.292556 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-df4245d4-3596-4e5f-b919-776cb2de4e3e tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:33.293034 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-df4245d4-3596-4e5f-b919-776cb2de4e3e tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:33.293705 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-df4245d4-3596-4e5f-b919-776cb2de4e3e tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:33.294551 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-df4245d4-3596-4e5f-b919-776cb2de4e3e tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:33.295101 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-df4245d4-3596-4e5f-b919-776cb2de4e3e tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:33.295767 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-df4245d4-3596-4e5f-b919-776cb2de4e3e tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:33.305973 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi [None req-f54603b7-99e3-492e-87c8-1ca1522b0fdd tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Action: 'action', calling method: >, body: {"shelve": {}} {{(pid=125640) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 19 00:42:33.306560 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.compute.api [None req-f54603b7-99e3-492e-87c8-1ca1522b0fdd tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] [instance: b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2] Fetching instance by UUID {{(pid=125640) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:42:33.312943 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-f54603b7-99e3-492e-87c8-1ca1522b0fdd tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:33.313634 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-f54603b7-99e3-492e-87c8-1ca1522b0fdd tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:33.313988 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-f54603b7-99e3-492e-87c8-1ca1522b0fdd tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:33.314752 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-f54603b7-99e3-492e-87c8-1ca1522b0fdd tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:33.315240 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-f54603b7-99e3-492e-87c8-1ca1522b0fdd tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:33.315669 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-f54603b7-99e3-492e-87c8-1ca1522b0fdd tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:33.316391 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.quota [None req-df4245d4-3596-4e5f-b919-776cb2de4e3e tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Getting quotas for project 173a52b79dab4962a4aa380c26f97d2d. Resources: {'cores', 'ram', 'instances'} {{(pid=125641) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 19 00:42:33.320503 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.quota [None req-df4245d4-3596-4e5f-b919-776cb2de4e3e tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Getting quotas for user a08c9457231a447eb1c4130e68cfb31d and project 173a52b79dab4962a4aa380c26f97d2d. Resources: {'cores', 'ram', 'instances'} {{(pid=125641) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 19 00:42:33.334452 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-df4245d4-3596-4e5f-b919-776cb2de4e3e tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Going to run 1 instances... {{(pid=125641) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 19 00:42:33.336882 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-df4245d4-3596-4e5f-b919-776cb2de4e3e tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] [instance: c5d02d64-d4e6-445d-a155-b875b8a8cce0] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='b387b4f8-f75e-4a68-b49c-a015bf61c46b',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=125641) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 19 00:42:33.340244 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.objects.instance [None req-f54603b7-99e3-492e-87c8-1ca1522b0fdd tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lazy-loading 'flavor' on Instance uuid b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2 {{(pid=125640) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 19 00:42:33.409463 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-df4245d4-3596-4e5f-b919-776cb2de4e3e tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 11.667895 Oct 19 00:42:33.410043 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 101/210] 10.222.0.26 () {60 vars in 1147 bytes} [Wed Oct 19 00:42:21 2022] POST /compute/v2.1/servers => generated 380 bytes in 11669 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 19 00:42:33.422032 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-6c303f8e-9de6-4779-967d-0bb0aa6bbddb tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:42:33.422858 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-6c303f8e-9de6-4779-967d-0bb0aa6bbddb tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] [instance: c5d02d64-d4e6-445d-a155-b875b8a8cce0] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:42:33.497046 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-6c303f8e-9de6-4779-967d-0bb0aa6bbddb tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a08c9457231a447eb1c4130e68cfb31d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '173a52b79dab4962a4aa380c26f97d2d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:42:33.508962 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-6c303f8e-9de6-4779-967d-0bb0aa6bbddb tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] 10.222.0.26 "GET /compute/v2.1/servers/c5d02d64-d4e6-445d-a155-b875b8a8cce0" status: 200 len: 1214 microversion: 2.1 time: 0.091749 Oct 19 00:42:33.522898 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 102/211] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:42:33 2022] GET /compute/v2.1/servers/c5d02d64-d4e6-445d-a155-b875b8a8cce0 => generated 1214 bytes in 106 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:42:33.615728 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: INFO nova.api.openstack.requestlog [None req-f54603b7-99e3-492e-87c8-1ca1522b0fdd tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] 10.222.0.26 "POST /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2/action" status: 202 len: 0 microversion: 2.20 time: 0.313412 Oct 19 00:42:33.616783 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: [pid: 125640|app: 0|req: 110/212] 10.222.0.26 () {62 vars in 1319 bytes} [Wed Oct 19 00:42:33 2022] POST /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2/action => generated 0 bytes in 315 msecs (HTTP/1.1 202) 9 headers in 363 bytes (2 switches on core 0) Oct 19 00:42:33.625981 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-ce94de82-4720-4c23-b689-50747489df4b tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:42:33.626690 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-ce94de82-4720-4c23-b689-50747489df4b tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] [instance: b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:42:33.633063 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ce94de82-4720-4c23-b689-50747489df4b tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:33.633542 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ce94de82-4720-4c23-b689-50747489df4b tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:33.633972 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ce94de82-4720-4c23-b689-50747489df4b tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:33.634710 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ce94de82-4720-4c23-b689-50747489df4b tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:33.635158 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ce94de82-4720-4c23-b689-50747489df4b tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:33.635577 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ce94de82-4720-4c23-b689-50747489df4b tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:33.789418 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-ce94de82-4720-4c23-b689-50747489df4b tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:42:33.800135 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-ce94de82-4720-4c23-b689-50747489df4b tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:42:33.802016 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-ce94de82-4720-4c23-b689-50747489df4b tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:42:33.803601 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-ce94de82-4720-4c23-b689-50747489df4b tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] 10.222.0.26 "GET /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2" status: 200 len: 1723 microversion: 2.20 time: 0.182561 Oct 19 00:42:33.804922 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 103/213] 10.222.0.26 () {60 vars in 1277 bytes} [Wed Oct 19 00:42:33 2022] GET /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2 => generated 1723 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 19 00:42:33.920966 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi [None req-d45fde18-5884-4934-a630-982cdd56a2de tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Calling method '>' {{(pid=125640) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:42:33.921660 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.compute.api [None req-d45fde18-5884-4934-a630-982cdd56a2de tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] [instance: 3da61834-fd67-49d2-925c-c18fc264042c] Fetching instance by UUID {{(pid=125640) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:42:33.927777 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-d45fde18-5884-4934-a630-982cdd56a2de tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:33.928257 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-d45fde18-5884-4934-a630-982cdd56a2de tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:33.928688 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-d45fde18-5884-4934-a630-982cdd56a2de tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:33.929371 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-d45fde18-5884-4934-a630-982cdd56a2de tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:33.929821 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-d45fde18-5884-4934-a630-982cdd56a2de tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:33.930243 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-d45fde18-5884-4934-a630-982cdd56a2de tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:34.067238 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.policy [None req-d45fde18-5884-4934-a630-982cdd56a2de tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd3f0d615187d4779bdd7da8c9f24c796', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0151d58abd047a692f5cc4e7d05bea5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125640) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:42:34.075737 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: INFO nova.api.openstack.requestlog [None req-d45fde18-5884-4934-a630-982cdd56a2de tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] 10.222.0.26 "GET /compute/v2.1/servers/3da61834-fd67-49d2-925c-c18fc264042c" status: 200 len: 1521 microversion: 2.1 time: 0.159762 Oct 19 00:42:34.076447 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: [pid: 125640|app: 0|req: 111/214] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:42:33 2022] GET /compute/v2.1/servers/3da61834-fd67-49d2-925c-c18fc264042c => generated 1521 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:42:34.534966 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-61ab7e7d-9b5b-457d-9f2a-427b3efc3716 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:42:34.535689 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-61ab7e7d-9b5b-457d-9f2a-427b3efc3716 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] [instance: c5d02d64-d4e6-445d-a155-b875b8a8cce0] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:42:34.542578 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-61ab7e7d-9b5b-457d-9f2a-427b3efc3716 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:34.543067 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-61ab7e7d-9b5b-457d-9f2a-427b3efc3716 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:34.543539 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-61ab7e7d-9b5b-457d-9f2a-427b3efc3716 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:34.544243 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-61ab7e7d-9b5b-457d-9f2a-427b3efc3716 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:34.544795 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-61ab7e7d-9b5b-457d-9f2a-427b3efc3716 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:34.546114 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-61ab7e7d-9b5b-457d-9f2a-427b3efc3716 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:34.621498 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-61ab7e7d-9b5b-457d-9f2a-427b3efc3716 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a08c9457231a447eb1c4130e68cfb31d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '173a52b79dab4962a4aa380c26f97d2d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:42:34.631486 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-61ab7e7d-9b5b-457d-9f2a-427b3efc3716 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] 10.222.0.26 "GET /compute/v2.1/servers/c5d02d64-d4e6-445d-a155-b875b8a8cce0" status: 200 len: 1210 microversion: 2.1 time: 0.102049 Oct 19 00:42:34.632077 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 104/215] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:42:34 2022] GET /compute/v2.1/servers/c5d02d64-d4e6-445d-a155-b875b8a8cce0 => generated 1210 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:42:34.836669 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi [None req-6fd4c612-e14f-4a4a-a608-a4da63ff025c tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Calling method '>' {{(pid=125640) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:42:34.837297 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.compute.api [None req-6fd4c612-e14f-4a4a-a608-a4da63ff025c tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] [instance: b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2] Fetching instance by UUID {{(pid=125640) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:42:34.846134 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-6fd4c612-e14f-4a4a-a608-a4da63ff025c tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:34.846949 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-6fd4c612-e14f-4a4a-a608-a4da63ff025c tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:34.847383 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-6fd4c612-e14f-4a4a-a608-a4da63ff025c tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:34.848455 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-6fd4c612-e14f-4a4a-a608-a4da63ff025c tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:34.848908 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-6fd4c612-e14f-4a4a-a608-a4da63ff025c tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:34.849328 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-6fd4c612-e14f-4a4a-a608-a4da63ff025c tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:35.005966 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.policy [None req-6fd4c612-e14f-4a4a-a608-a4da63ff025c tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125640) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:42:35.014973 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.policy [None req-6fd4c612-e14f-4a4a-a608-a4da63ff025c tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125640) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:42:35.016590 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.policy [None req-6fd4c612-e14f-4a4a-a608-a4da63ff025c tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125640) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:42:35.018122 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: INFO nova.api.openstack.requestlog [None req-6fd4c612-e14f-4a4a-a608-a4da63ff025c tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] 10.222.0.26 "GET /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2" status: 200 len: 1723 microversion: 2.20 time: 0.185237 Oct 19 00:42:35.026606 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: [pid: 125640|app: 0|req: 112/216] 10.222.0.26 () {60 vars in 1277 bytes} [Wed Oct 19 00:42:34 2022] GET /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2 => generated 1723 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 19 00:42:35.102429 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-ed4eb5a2-d85b-48dd-bd23-c6363ae2f69f tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:42:35.103734 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-ed4eb5a2-d85b-48dd-bd23-c6363ae2f69f tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] [instance: 3da61834-fd67-49d2-925c-c18fc264042c] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:42:35.110093 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ed4eb5a2-d85b-48dd-bd23-c6363ae2f69f tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:35.110659 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ed4eb5a2-d85b-48dd-bd23-c6363ae2f69f tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:35.111125 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ed4eb5a2-d85b-48dd-bd23-c6363ae2f69f tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:35.111825 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ed4eb5a2-d85b-48dd-bd23-c6363ae2f69f tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:35.112287 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ed4eb5a2-d85b-48dd-bd23-c6363ae2f69f tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:35.112734 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ed4eb5a2-d85b-48dd-bd23-c6363ae2f69f tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:35.253812 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-ed4eb5a2-d85b-48dd-bd23-c6363ae2f69f tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd3f0d615187d4779bdd7da8c9f24c796', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0151d58abd047a692f5cc4e7d05bea5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:42:35.263755 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-ed4eb5a2-d85b-48dd-bd23-c6363ae2f69f tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] 10.222.0.26 "GET /compute/v2.1/servers/3da61834-fd67-49d2-925c-c18fc264042c" status: 200 len: 1521 microversion: 2.1 time: 0.166281 Oct 19 00:42:35.264905 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 105/217] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:42:35 2022] GET /compute/v2.1/servers/3da61834-fd67-49d2-925c-c18fc264042c => generated 1521 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 19 00:42:35.655189 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi [None req-753a2a76-098f-4a56-8a76-cdaf6cf0f309 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Calling method '>' {{(pid=125640) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:42:35.656165 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.compute.api [None req-753a2a76-098f-4a56-8a76-cdaf6cf0f309 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] [instance: c5d02d64-d4e6-445d-a155-b875b8a8cce0] Fetching instance by UUID {{(pid=125640) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:42:35.662789 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-753a2a76-098f-4a56-8a76-cdaf6cf0f309 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:35.663275 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-753a2a76-098f-4a56-8a76-cdaf6cf0f309 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:35.663717 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-753a2a76-098f-4a56-8a76-cdaf6cf0f309 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:35.664594 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-753a2a76-098f-4a56-8a76-cdaf6cf0f309 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:35.665083 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-753a2a76-098f-4a56-8a76-cdaf6cf0f309 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:35.665510 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-753a2a76-098f-4a56-8a76-cdaf6cf0f309 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:35.772463 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.policy [None req-753a2a76-098f-4a56-8a76-cdaf6cf0f309 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a08c9457231a447eb1c4130e68cfb31d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '173a52b79dab4962a4aa380c26f97d2d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125640) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:42:35.780445 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: INFO nova.api.openstack.requestlog [None req-753a2a76-098f-4a56-8a76-cdaf6cf0f309 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] 10.222.0.26 "GET /compute/v2.1/servers/c5d02d64-d4e6-445d-a155-b875b8a8cce0" status: 200 len: 1274 microversion: 2.1 time: 0.129177 Oct 19 00:42:35.781477 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: [pid: 125640|app: 0|req: 113/218] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:42:35 2022] GET /compute/v2.1/servers/c5d02d64-d4e6-445d-a155-b875b8a8cce0 => generated 1274 bytes in 131 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:42:36.045540 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-a65a937e-fe7f-4b3c-a9d2-c1cabe62379f tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:42:36.046192 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-a65a937e-fe7f-4b3c-a9d2-c1cabe62379f tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] [instance: b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:42:36.052199 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-a65a937e-fe7f-4b3c-a9d2-c1cabe62379f tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:36.052683 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-a65a937e-fe7f-4b3c-a9d2-c1cabe62379f tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:36.053123 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-a65a937e-fe7f-4b3c-a9d2-c1cabe62379f tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:36.053828 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-a65a937e-fe7f-4b3c-a9d2-c1cabe62379f tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:36.054356 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-a65a937e-fe7f-4b3c-a9d2-c1cabe62379f tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:36.054887 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-a65a937e-fe7f-4b3c-a9d2-c1cabe62379f tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:36.193112 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-a65a937e-fe7f-4b3c-a9d2-c1cabe62379f tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:42:36.201269 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-a65a937e-fe7f-4b3c-a9d2-c1cabe62379f tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:42:36.202817 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-a65a937e-fe7f-4b3c-a9d2-c1cabe62379f tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:42:36.204103 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-a65a937e-fe7f-4b3c-a9d2-c1cabe62379f tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] 10.222.0.26 "GET /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2" status: 200 len: 1723 microversion: 2.20 time: 0.162884 Oct 19 00:42:36.204912 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 106/219] 10.222.0.26 () {60 vars in 1277 bytes} [Wed Oct 19 00:42:36 2022] GET /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2 => generated 1723 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 19 00:42:36.286341 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi [None req-50b7e6fd-8a60-4e5b-a01f-8346a44d5b6f tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Calling method '>' {{(pid=125640) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:42:36.286992 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.compute.api [None req-50b7e6fd-8a60-4e5b-a01f-8346a44d5b6f tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] [instance: 3da61834-fd67-49d2-925c-c18fc264042c] Fetching instance by UUID {{(pid=125640) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:42:36.289454 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG dbcounter [-] [125640] Writing DB stats nova_api:SELECT=67,nova_api:UPDATE=1 {{(pid=125640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 00:42:36.294889 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-50b7e6fd-8a60-4e5b-a01f-8346a44d5b6f tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:36.295929 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-50b7e6fd-8a60-4e5b-a01f-8346a44d5b6f tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:36.296310 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-50b7e6fd-8a60-4e5b-a01f-8346a44d5b6f tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:36.297020 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-50b7e6fd-8a60-4e5b-a01f-8346a44d5b6f tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:36.297472 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-50b7e6fd-8a60-4e5b-a01f-8346a44d5b6f tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:36.297960 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-50b7e6fd-8a60-4e5b-a01f-8346a44d5b6f tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:36.438773 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.policy [None req-50b7e6fd-8a60-4e5b-a01f-8346a44d5b6f tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd3f0d615187d4779bdd7da8c9f24c796', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0151d58abd047a692f5cc4e7d05bea5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125640) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:42:36.446603 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: INFO nova.api.openstack.requestlog [None req-50b7e6fd-8a60-4e5b-a01f-8346a44d5b6f tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] 10.222.0.26 "GET /compute/v2.1/servers/3da61834-fd67-49d2-925c-c18fc264042c" status: 200 len: 1521 microversion: 2.1 time: 0.164431 Oct 19 00:42:36.447171 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: [pid: 125640|app: 0|req: 114/220] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:42:36 2022] GET /compute/v2.1/servers/3da61834-fd67-49d2-925c-c18fc264042c => generated 1521 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:42:36.805833 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-7220e24a-3c65-4924-b141-5ad25543f793 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:42:36.806576 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-7220e24a-3c65-4924-b141-5ad25543f793 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] [instance: c5d02d64-d4e6-445d-a155-b875b8a8cce0] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:42:36.813411 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-7220e24a-3c65-4924-b141-5ad25543f793 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:36.814007 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-7220e24a-3c65-4924-b141-5ad25543f793 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:36.815095 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-7220e24a-3c65-4924-b141-5ad25543f793 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:36.815907 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-7220e24a-3c65-4924-b141-5ad25543f793 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:36.816426 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-7220e24a-3c65-4924-b141-5ad25543f793 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:36.817116 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-7220e24a-3c65-4924-b141-5ad25543f793 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:36.929148 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-7220e24a-3c65-4924-b141-5ad25543f793 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a08c9457231a447eb1c4130e68cfb31d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '173a52b79dab4962a4aa380c26f97d2d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:42:36.940259 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-7220e24a-3c65-4924-b141-5ad25543f793 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] 10.222.0.26 "GET /compute/v2.1/servers/c5d02d64-d4e6-445d-a155-b875b8a8cce0" status: 200 len: 1272 microversion: 2.1 time: 0.139444 Oct 19 00:42:36.941187 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 107/221] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:42:36 2022] GET /compute/v2.1/servers/c5d02d64-d4e6-445d-a155-b875b8a8cce0 => generated 1272 bytes in 141 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:42:37.230615 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi [None req-8d6711e7-d8a4-4810-af05-8a66e1fbecc3 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Calling method '>' {{(pid=125640) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:42:37.231252 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.compute.api [None req-8d6711e7-d8a4-4810-af05-8a66e1fbecc3 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] [instance: b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2] Fetching instance by UUID {{(pid=125640) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:42:37.237430 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-8d6711e7-d8a4-4810-af05-8a66e1fbecc3 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:37.237923 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-8d6711e7-d8a4-4810-af05-8a66e1fbecc3 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:37.238363 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-8d6711e7-d8a4-4810-af05-8a66e1fbecc3 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:37.239103 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-8d6711e7-d8a4-4810-af05-8a66e1fbecc3 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:37.239568 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-8d6711e7-d8a4-4810-af05-8a66e1fbecc3 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:37.239999 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG oslo_concurrency.lockutils [None req-8d6711e7-d8a4-4810-af05-8a66e1fbecc3 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125640) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:37.456569 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.policy [None req-8d6711e7-d8a4-4810-af05-8a66e1fbecc3 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125640) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:42:37.466801 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.policy [None req-8d6711e7-d8a4-4810-af05-8a66e1fbecc3 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125640) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:42:37.469277 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.policy [None req-8d6711e7-d8a4-4810-af05-8a66e1fbecc3 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125640) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:42:37.469921 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-d2569283-9aac-4d85-8400-30f3b011dded tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:42:37.471156 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-d2569283-9aac-4d85-8400-30f3b011dded tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] [instance: 3da61834-fd67-49d2-925c-c18fc264042c] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:42:37.471520 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: INFO nova.api.openstack.requestlog [None req-8d6711e7-d8a4-4810-af05-8a66e1fbecc3 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] 10.222.0.26 "GET /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2" status: 200 len: 1723 microversion: 2.20 time: 0.244463 Oct 19 00:42:37.472317 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: [pid: 125640|app: 0|req: 115/222] 10.222.0.26 () {60 vars in 1277 bytes} [Wed Oct 19 00:42:37 2022] GET /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2 => generated 1723 bytes in 246 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 19 00:42:37.478256 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d2569283-9aac-4d85-8400-30f3b011dded tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:37.478792 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d2569283-9aac-4d85-8400-30f3b011dded tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:37.479565 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d2569283-9aac-4d85-8400-30f3b011dded tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:37.480278 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d2569283-9aac-4d85-8400-30f3b011dded tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:37.481031 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d2569283-9aac-4d85-8400-30f3b011dded tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:37.481468 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d2569283-9aac-4d85-8400-30f3b011dded tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:37.640972 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-d2569283-9aac-4d85-8400-30f3b011dded tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd3f0d615187d4779bdd7da8c9f24c796', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0151d58abd047a692f5cc4e7d05bea5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:42:37.649812 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.api.openstack.wsgi [None req-91679b98-bde7-4d43-a4f4-037cbef9f908 tempest-TestShelveInstance-770582455 tempest-TestShelveInstance-770582455-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestShelveInstance-server-1227343997", "imageRef": "", "flavorRef": "452", "key_name": "tempest-TestShelveInstance-480977406", "security_groups": [{"name": "tempest-secgroup-smoke-1086442532"}], "networks": [{"uuid": "d812fe1e-848c-4fe4-a621-f8ac43794710"}], "block_device_mapping_v2": [{"uuid": "24e9decf-0b95-46d1-b766-f7b01125c78d", "source_type": "volume", "destination_type": "volume", "boot_index": 0, "delete_on_termination": true}]}} {{(pid=125640) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 19 00:42:37.650062 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-d2569283-9aac-4d85-8400-30f3b011dded tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] 10.222.0.26 "GET /compute/v2.1/servers/3da61834-fd67-49d2-925c-c18fc264042c" status: 200 len: 1521 microversion: 2.1 time: 0.184691 Oct 19 00:42:37.652722 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 108/223] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:42:37 2022] GET /compute/v2.1/servers/3da61834-fd67-49d2-925c-c18fc264042c => generated 1521 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:42:37.768273 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG nova.network.neutron [None req-91679b98-bde7-4d43-a4f4-037cbef9f908 tempest-TestShelveInstance-770582455 tempest-TestShelveInstance-770582455-project] validate_networks() for [('d812fe1e-848c-4fe4-a621-f8ac43794710', None, None, None, None, None)] {{(pid=125640) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 19 00:42:37.964778 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-9f552edb-131a-4241-a705-cdc9f07eb4b7 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:42:37.965712 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-9f552edb-131a-4241-a705-cdc9f07eb4b7 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] [instance: c5d02d64-d4e6-445d-a155-b875b8a8cce0] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:42:37.972682 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-9f552edb-131a-4241-a705-cdc9f07eb4b7 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:37.973316 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-9f552edb-131a-4241-a705-cdc9f07eb4b7 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:37.973752 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-9f552edb-131a-4241-a705-cdc9f07eb4b7 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:37.974444 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-9f552edb-131a-4241-a705-cdc9f07eb4b7 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:37.974935 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-9f552edb-131a-4241-a705-cdc9f07eb4b7 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:37.975380 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-9f552edb-131a-4241-a705-cdc9f07eb4b7 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:38.121179 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-9f552edb-131a-4241-a705-cdc9f07eb4b7 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a08c9457231a447eb1c4130e68cfb31d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '173a52b79dab4962a4aa380c26f97d2d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:42:38.130227 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-9f552edb-131a-4241-a705-cdc9f07eb4b7 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] 10.222.0.26 "GET /compute/v2.1/servers/c5d02d64-d4e6-445d-a155-b875b8a8cce0" status: 200 len: 1314 microversion: 2.1 time: 0.169252 Oct 19 00:42:38.146607 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 109/224] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:42:37 2022] GET /compute/v2.1/servers/c5d02d64-d4e6-445d-a155-b875b8a8cce0 => generated 1314 bytes in 186 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 19 00:42:38.481493 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [req-c4544a82-f530-4d8b-9363-e2f34fbb0903 req-5581e870-65da-4cfa-823f-67f8fefadc13 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "c5d02d64-d4e6-445d-a155-b875b8a8cce0", "tag": "1b4d823f-e9a3-4a43-a759-c283b8c3dac4"}]} {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 19 00:42:38.492037 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-c4544a82-f530-4d8b-9363-e2f34fbb0903 req-5581e870-65da-4cfa-823f-67f8fefadc13 service nova] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:38.492519 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-c4544a82-f530-4d8b-9363-e2f34fbb0903 req-5581e870-65da-4cfa-823f-67f8fefadc13 service nova] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:38.492954 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-c4544a82-f530-4d8b-9363-e2f34fbb0903 req-5581e870-65da-4cfa-823f-67f8fefadc13 service nova] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:38.504676 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.compute.server_external_events [req-c4544a82-f530-4d8b-9363-e2f34fbb0903 req-5581e870-65da-4cfa-823f-67f8fefadc13 service nova] Creating event network-changed:1b4d823f-e9a3-4a43-a759-c283b8c3dac4 for instance c5d02d64-d4e6-445d-a155-b875b8a8cce0 on hv-cinder-maste Oct 19 00:42:38.509979 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [req-c4544a82-f530-4d8b-9363-e2f34fbb0903 req-5581e870-65da-4cfa-823f-67f8fefadc13 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.032280 Oct 19 00:42:38.510901 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 110/225] 10.222.0.26 () {64 vars in 1317 bytes} [Wed Oct 19 00:42:38 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 19 00:42:38.519488 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-0314c622-a662-4a17-bd94-384bd21a9c5b tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:42:38.520184 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-0314c622-a662-4a17-bd94-384bd21a9c5b tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] [instance: b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:42:38.526295 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-0314c622-a662-4a17-bd94-384bd21a9c5b tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:38.526817 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-0314c622-a662-4a17-bd94-384bd21a9c5b tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:38.527254 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-0314c622-a662-4a17-bd94-384bd21a9c5b tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:38.527961 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-0314c622-a662-4a17-bd94-384bd21a9c5b tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:38.528413 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-0314c622-a662-4a17-bd94-384bd21a9c5b tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:38.528835 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-0314c622-a662-4a17-bd94-384bd21a9c5b tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:38.681238 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-0314c622-a662-4a17-bd94-384bd21a9c5b tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:42:38.691998 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-0314c622-a662-4a17-bd94-384bd21a9c5b tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:42:38.693566 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-0314c622-a662-4a17-bd94-384bd21a9c5b tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:42:38.694871 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-0314c622-a662-4a17-bd94-384bd21a9c5b tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] 10.222.0.26 "GET /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2" status: 200 len: 1723 microversion: 2.20 time: 0.182648 Oct 19 00:42:38.695496 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 111/226] 10.222.0.26 () {60 vars in 1277 bytes} [Wed Oct 19 00:42:38 2022] GET /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2 => generated 1723 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 19 00:42:38.700797 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-a9dd9fbd-2782-42a6-bdad-7e35f7080fa0 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:42:38.701435 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-a9dd9fbd-2782-42a6-bdad-7e35f7080fa0 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] [instance: 3da61834-fd67-49d2-925c-c18fc264042c] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:42:38.707530 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-a9dd9fbd-2782-42a6-bdad-7e35f7080fa0 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:38.708151 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-a9dd9fbd-2782-42a6-bdad-7e35f7080fa0 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:38.708592 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-a9dd9fbd-2782-42a6-bdad-7e35f7080fa0 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:38.709621 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-a9dd9fbd-2782-42a6-bdad-7e35f7080fa0 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:38.710227 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-a9dd9fbd-2782-42a6-bdad-7e35f7080fa0 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:38.710839 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-a9dd9fbd-2782-42a6-bdad-7e35f7080fa0 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:38.853356 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-a9dd9fbd-2782-42a6-bdad-7e35f7080fa0 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd3f0d615187d4779bdd7da8c9f24c796', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0151d58abd047a692f5cc4e7d05bea5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:42:38.862411 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-a9dd9fbd-2782-42a6-bdad-7e35f7080fa0 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] 10.222.0.26 "GET /compute/v2.1/servers/3da61834-fd67-49d2-925c-c18fc264042c" status: 200 len: 1501 microversion: 2.1 time: 0.165739 Oct 19 00:42:38.863015 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 112/227] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:42:38 2022] GET /compute/v2.1/servers/3da61834-fd67-49d2-925c-c18fc264042c => generated 1501 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:42:38.885940 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-954c8d73-9ffd-47db-96f0-2fa117708592 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "d1e5c042-6ee4-4a71-8f25-ff1c3f78ea8f"}} {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 19 00:42:38.888945 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-954c8d73-9ffd-47db-96f0-2fa117708592 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] [instance: 3da61834-fd67-49d2-925c-c18fc264042c] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:42:38.895250 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-954c8d73-9ffd-47db-96f0-2fa117708592 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:38.895730 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-954c8d73-9ffd-47db-96f0-2fa117708592 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:38.896164 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-954c8d73-9ffd-47db-96f0-2fa117708592 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:38.896890 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-954c8d73-9ffd-47db-96f0-2fa117708592 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:38.897328 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-954c8d73-9ffd-47db-96f0-2fa117708592 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:38.897742 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-954c8d73-9ffd-47db-96f0-2fa117708592 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:38.924899 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.wsgi [None req-954c8d73-9ffd-47db-96f0-2fa117708592 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] HTTP exception thrown: Cannot 'attach_volume' instance 3da61834-fd67-49d2-925c-c18fc264042c while it is in vm_state rescued Oct 19 00:42:38.926634 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-954c8d73-9ffd-47db-96f0-2fa117708592 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Request method failure captured: Oct 19 00:42:38.926634 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: request: POST /compute/v2.1/servers/3da61834-fd67-49d2-925c-c18fc264042c/os-volume_attachments HTTP/1.1 Oct 19 00:42:38.926634 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Accept: application/json Oct 19 00:42:38.926634 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Accept-Encoding: identity Oct 19 00:42:38.926634 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Connection: close Oct 19 00:42:38.926634 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Content-Length: 74 Oct 19 00:42:38.926634 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Content-Type: application/json Oct 19 00:42:38.926634 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Host: 10.222.0.26 Oct 19 00:42:38.926634 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Openstack-System-Scope: None Oct 19 00:42:38.926634 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: User-Agent: python-urllib3/1.26.12 Oct 19 00:42:38.926634 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Auth-Token: gAAAAABjT0eG-h4KFbZ30OiVtvJ2IwzaIfjKNSU1Qsat-O9_Zg1Ouctw-AlAuBWD6kHGuP3EnbU_TJPE1ljWwW1kJo03Mn42ZE5duN2HDm9MXxFr1Oc_Mse4KR4ABtf0tvzP53Pz6nUn07toZIxQKWNLC4yurNeR_6sS4VV90-4qptRCmdstdYU Oct 19 00:42:38.926634 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Domain-Id: None Oct 19 00:42:38.926634 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Domain-Name: None Oct 19 00:42:38.926634 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Identity-Status: Confirmed Oct 19 00:42:38.926634 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Is-Admin-Project: True Oct 19 00:42:38.926634 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Project-Domain-Id: default Oct 19 00:42:38.926634 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Project-Domain-Name: Default Oct 19 00:42:38.926634 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Project-Id: c0151d58abd047a692f5cc4e7d05bea5 Oct 19 00:42:38.926634 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-190719564 Oct 19 00:42:38.926634 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Role: member,reader Oct 19 00:42:38.926634 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Roles: member,reader Oct 19 00:42:38.928345 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/c0151d58abd047a692f5cc4e7d05bea5"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/c0151d58abd047a692f5cc4e7d05bea5"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/c0151d58abd047a692f5cc4e7d05bea5"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_c0151d58abd047a692f5cc4e7d05bea5"}]}] Oct 19 00:42:38.928345 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Tenant: tempest-ServerRescueNegativeTestJSON-190719564 Oct 19 00:42:38.928345 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Tenant-Id: c0151d58abd047a692f5cc4e7d05bea5 Oct 19 00:42:38.928345 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-190719564 Oct 19 00:42:38.928345 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User: tempest-ServerRescueNegativeTestJSON-190719564-project Oct 19 00:42:38.928345 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User-Domain-Id: default Oct 19 00:42:38.928345 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User-Domain-Name: Default Oct 19 00:42:38.928345 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User-Id: d3f0d615187d4779bdd7da8c9f24c796 Oct 19 00:42:38.928345 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User-Name: tempest-ServerRescueNegativeTestJSON-190719564-project Oct 19 00:42:38.928345 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Oct 19 00:42:38.928345 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: {"volumeAttachment": {"volumeId": "d1e5c042-6ee4-4a71-8f25-ff1c3f78ea8f"}} Oct 19 00:42:38.928345 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: method: > Oct 19 00:42:38.929177 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: exception: Cannot 'attach_volume' instance 3da61834-fd67-49d2-925c-c18fc264042c while it is in vm_state rescued Oct 19 00:42:38.929177 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: action_args: {'server_id': '3da61834-fd67-49d2-925c-c18fc264042c', 'body': {'volumeAttachment': {'volumeId': 'd1e5c042-6ee4-4a71-8f25-ff1c3f78ea8f'}}} Oct 19 00:42:38.929177 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 19 00:42:38.929177 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 19 00:42:38.929177 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 388, in create Oct 19 00:42:38.929177 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi device = self.compute_api.attach_volume( Oct 19 00:42:38.929177 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 241, in inner Oct 19 00:42:38.929177 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Oct 19 00:42:38.929177 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 148, in inner Oct 19 00:42:38.929177 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi raise exception.InstanceInvalidState( Oct 19 00:42:38.929177 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi nova.exception.InstanceInvalidState: Instance 3da61834-fd67-49d2-925c-c18fc264042c in vm_state rescued. Cannot attach_volume while the instance is in this state. Oct 19 00:42:38.929177 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:42:38.929177 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 19 00:42:38.929177 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:42:38.929177 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 19 00:42:38.929177 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 19 00:42:38.929177 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 19 00:42:38.929177 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 19 00:42:38.929177 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 19 00:42:38.930452 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 19 00:42:38.930452 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 19 00:42:38.930452 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 19 00:42:38.930452 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 19 00:42:38.930452 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 19 00:42:38.930452 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 19 00:42:38.930452 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 19 00:42:38.930452 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 19 00:42:38.930452 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 398, in create Oct 19 00:42:38.930452 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi common.raise_http_conflict_for_instance_invalid_state(state_error, Oct 19 00:42:38.930452 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 347, in raise_http_conflict_for_instance_invalid_state Oct 19 00:42:38.930452 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPConflict(explanation=msg) Oct 19 00:42:38.930452 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi webob.exc.HTTPConflict: Cannot 'attach_volume' instance 3da61834-fd67-49d2-925c-c18fc264042c while it is in vm_state rescued Oct 19 00:42:38.930452 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:42:38.930452 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 19 00:42:38.930452 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:42:38.930452 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 19 00:42:38.930452 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 19 00:42:38.930452 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 19 00:42:38.931794 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 19 00:42:38.931794 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 19 00:42:38.931794 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Cannot 'attach_volume' instance 3da61834-fd67-49d2-925c-c18fc264042c while it is in vm_state rescued Oct 19 00:42:38.931794 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:42:38.931794 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-954c8d73-9ffd-47db-96f0-2fa117708592 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Returning 409 to user: Cannot 'attach_volume' instance 3da61834-fd67-49d2-925c-c18fc264042c while it is in vm_state rescued {{(pid=125641) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 19 00:42:38.931794 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-954c8d73-9ffd-47db-96f0-2fa117708592 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] 10.222.0.26 "POST /compute/v2.1/servers/3da61834-fd67-49d2-925c-c18fc264042c/os-volume_attachments" status: 409 len: 152 microversion: 2.1 time: 0.046441 Oct 19 00:42:38.932625 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 113/228] 10.222.0.26 () {60 vars in 1323 bytes} [Wed Oct 19 00:42:38 2022] POST /compute/v2.1/servers/3da61834-fd67-49d2-925c-c18fc264042c/os-volume_attachments => generated 152 bytes in 51 msecs (HTTP/1.1 409) 9 headers in 378 bytes (2 switches on core 0) Oct 19 00:42:38.937579 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-e86ba816-3be2-4a93-b60a-8a7f25fe46ad tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Action: 'action', calling method: >, body: {"unrescue": {}} {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 19 00:42:38.938112 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-e86ba816-3be2-4a93-b60a-8a7f25fe46ad tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] [instance: 3da61834-fd67-49d2-925c-c18fc264042c] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:42:38.944023 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-e86ba816-3be2-4a93-b60a-8a7f25fe46ad tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:38.944514 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-e86ba816-3be2-4a93-b60a-8a7f25fe46ad tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:38.944953 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-e86ba816-3be2-4a93-b60a-8a7f25fe46ad tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:38.945642 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-e86ba816-3be2-4a93-b60a-8a7f25fe46ad tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:38.946093 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-e86ba816-3be2-4a93-b60a-8a7f25fe46ad tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:38.946548 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-e86ba816-3be2-4a93-b60a-8a7f25fe46ad tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:39.021242 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-e86ba816-3be2-4a93-b60a-8a7f25fe46ad tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] 10.222.0.26 "POST /compute/v2.1/servers/3da61834-fd67-49d2-925c-c18fc264042c/action" status: 202 len: 0 microversion: 2.1 time: 0.087360 Oct 19 00:42:39.024845 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 114/229] 10.222.0.26 () {60 vars in 1278 bytes} [Wed Oct 19 00:42:38 2022] POST /compute/v2.1/servers/3da61834-fd67-49d2-925c-c18fc264042c/action => generated 0 bytes in 89 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 19 00:42:39.030803 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-04b3de37-c538-4269-ba7f-378f641e304e tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:42:39.031703 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-04b3de37-c538-4269-ba7f-378f641e304e tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] [instance: 3da61834-fd67-49d2-925c-c18fc264042c] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:42:39.037782 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-04b3de37-c538-4269-ba7f-378f641e304e tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:39.038257 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-04b3de37-c538-4269-ba7f-378f641e304e tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:39.038716 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-04b3de37-c538-4269-ba7f-378f641e304e tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:39.039533 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-04b3de37-c538-4269-ba7f-378f641e304e tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:39.039945 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-04b3de37-c538-4269-ba7f-378f641e304e tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:39.040381 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-04b3de37-c538-4269-ba7f-378f641e304e tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:39.195130 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-04b3de37-c538-4269-ba7f-378f641e304e tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd3f0d615187d4779bdd7da8c9f24c796', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0151d58abd047a692f5cc4e7d05bea5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:42:39.205374 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-04b3de37-c538-4269-ba7f-378f641e304e tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] 10.222.0.26 "GET /compute/v2.1/servers/3da61834-fd67-49d2-925c-c18fc264042c" status: 200 len: 1509 microversion: 2.1 time: 0.178757 Oct 19 00:42:39.206030 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 115/230] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:42:39 2022] GET /compute/v2.1/servers/3da61834-fd67-49d2-925c-c18fc264042c => generated 1509 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:42:39.215861 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-0b4b8545-ac99-4e3b-8bc6-ea0a0637178f tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:42:39.216467 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-0b4b8545-ac99-4e3b-8bc6-ea0a0637178f tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] [instance: c5d02d64-d4e6-445d-a155-b875b8a8cce0] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:42:39.223703 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-0b4b8545-ac99-4e3b-8bc6-ea0a0637178f tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:39.224184 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-0b4b8545-ac99-4e3b-8bc6-ea0a0637178f tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:39.224641 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-0b4b8545-ac99-4e3b-8bc6-ea0a0637178f tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:39.225328 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-0b4b8545-ac99-4e3b-8bc6-ea0a0637178f tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:39.225780 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-0b4b8545-ac99-4e3b-8bc6-ea0a0637178f tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:39.226201 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-0b4b8545-ac99-4e3b-8bc6-ea0a0637178f tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:39.371326 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-0b4b8545-ac99-4e3b-8bc6-ea0a0637178f tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a08c9457231a447eb1c4130e68cfb31d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '173a52b79dab4962a4aa380c26f97d2d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:42:39.380049 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-0b4b8545-ac99-4e3b-8bc6-ea0a0637178f tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] 10.222.0.26 "GET /compute/v2.1/servers/c5d02d64-d4e6-445d-a155-b875b8a8cce0" status: 200 len: 1314 microversion: 2.1 time: 0.167966 Oct 19 00:42:39.380629 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 116/231] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:42:39 2022] GET /compute/v2.1/servers/c5d02d64-d4e6-445d-a155-b875b8a8cce0 => generated 1314 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:42:39.723803 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-c4caade2-6295-4345-9561-aaaa00a0b5b5 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:42:39.724453 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-c4caade2-6295-4345-9561-aaaa00a0b5b5 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] [instance: b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:42:39.730376 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-c4caade2-6295-4345-9561-aaaa00a0b5b5 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:39.730892 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-c4caade2-6295-4345-9561-aaaa00a0b5b5 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:39.731374 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-c4caade2-6295-4345-9561-aaaa00a0b5b5 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:39.732063 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-c4caade2-6295-4345-9561-aaaa00a0b5b5 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:39.732512 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-c4caade2-6295-4345-9561-aaaa00a0b5b5 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:39.733513 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-c4caade2-6295-4345-9561-aaaa00a0b5b5 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:39.867048 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-c4caade2-6295-4345-9561-aaaa00a0b5b5 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:42:39.875036 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-c4caade2-6295-4345-9561-aaaa00a0b5b5 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:42:39.876465 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-c4caade2-6295-4345-9561-aaaa00a0b5b5 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:42:39.877606 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-c4caade2-6295-4345-9561-aaaa00a0b5b5 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] 10.222.0.26 "GET /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2" status: 200 len: 1723 microversion: 2.20 time: 0.158158 Oct 19 00:42:39.878154 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 117/232] 10.222.0.26 () {60 vars in 1277 bytes} [Wed Oct 19 00:42:39 2022] GET /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2 => generated 1723 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 19 00:42:40.230456 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-2f614335-1516-4305-b150-1e9e959c8cec tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:42:40.231093 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-2f614335-1516-4305-b150-1e9e959c8cec tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] [instance: 3da61834-fd67-49d2-925c-c18fc264042c] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:42:40.237705 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-2f614335-1516-4305-b150-1e9e959c8cec tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:40.238192 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-2f614335-1516-4305-b150-1e9e959c8cec tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:40.238669 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-2f614335-1516-4305-b150-1e9e959c8cec tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:40.239365 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-2f614335-1516-4305-b150-1e9e959c8cec tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:40.239826 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-2f614335-1516-4305-b150-1e9e959c8cec tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:40.240246 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-2f614335-1516-4305-b150-1e9e959c8cec tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:40.370656 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-2f614335-1516-4305-b150-1e9e959c8cec tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd3f0d615187d4779bdd7da8c9f24c796', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0151d58abd047a692f5cc4e7d05bea5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:42:40.378952 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-2f614335-1516-4305-b150-1e9e959c8cec tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] 10.222.0.26 "GET /compute/v2.1/servers/3da61834-fd67-49d2-925c-c18fc264042c" status: 200 len: 1509 microversion: 2.1 time: 0.152416 Oct 19 00:42:40.379516 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 118/233] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:42:40 2022] GET /compute/v2.1/servers/3da61834-fd67-49d2-925c-c18fc264042c => generated 1509 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 19 00:42:40.403165 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-c5cf847c-71b7-4cba-85c4-ca4eadcdb585 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:42:40.403798 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-c5cf847c-71b7-4cba-85c4-ca4eadcdb585 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] [instance: c5d02d64-d4e6-445d-a155-b875b8a8cce0] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:42:40.409618 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-c5cf847c-71b7-4cba-85c4-ca4eadcdb585 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:40.410099 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-c5cf847c-71b7-4cba-85c4-ca4eadcdb585 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:40.410553 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-c5cf847c-71b7-4cba-85c4-ca4eadcdb585 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:40.411238 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-c5cf847c-71b7-4cba-85c4-ca4eadcdb585 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:40.411688 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-c5cf847c-71b7-4cba-85c4-ca4eadcdb585 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:40.412109 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-c5cf847c-71b7-4cba-85c4-ca4eadcdb585 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:40.416257 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG dbcounter [-] [125641] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=125641) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 00:42:40.556133 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-c5cf847c-71b7-4cba-85c4-ca4eadcdb585 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a08c9457231a447eb1c4130e68cfb31d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '173a52b79dab4962a4aa380c26f97d2d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:42:40.565666 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-c5cf847c-71b7-4cba-85c4-ca4eadcdb585 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] 10.222.0.26 "GET /compute/v2.1/servers/c5d02d64-d4e6-445d-a155-b875b8a8cce0" status: 200 len: 1314 microversion: 2.1 time: 0.166095 Oct 19 00:42:40.566228 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 119/234] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:42:40 2022] GET /compute/v2.1/servers/c5d02d64-d4e6-445d-a155-b875b8a8cce0 => generated 1314 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 19 00:42:40.902580 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-2fa4bb68-3011-42ce-a006-87906a889a91 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:42:40.903188 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-2fa4bb68-3011-42ce-a006-87906a889a91 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] [instance: b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:42:40.909627 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-2fa4bb68-3011-42ce-a006-87906a889a91 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:40.910103 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-2fa4bb68-3011-42ce-a006-87906a889a91 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:40.910572 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-2fa4bb68-3011-42ce-a006-87906a889a91 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:40.911269 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-2fa4bb68-3011-42ce-a006-87906a889a91 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:40.911707 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-2fa4bb68-3011-42ce-a006-87906a889a91 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:40.912123 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-2fa4bb68-3011-42ce-a006-87906a889a91 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:41.044991 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-2fa4bb68-3011-42ce-a006-87906a889a91 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:42:41.053331 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-2fa4bb68-3011-42ce-a006-87906a889a91 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:42:41.054793 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-2fa4bb68-3011-42ce-a006-87906a889a91 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:42:41.055954 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-2fa4bb68-3011-42ce-a006-87906a889a91 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] 10.222.0.26 "GET /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2" status: 200 len: 1723 microversion: 2.20 time: 0.157442 Oct 19 00:42:41.056530 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 120/235] 10.222.0.26 () {60 vars in 1277 bytes} [Wed Oct 19 00:42:40 2022] GET /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2 => generated 1723 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 19 00:42:41.404148 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-1e918dea-71a1-4977-9cb3-3058415f0325 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:42:41.405042 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-1e918dea-71a1-4977-9cb3-3058415f0325 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] [instance: 3da61834-fd67-49d2-925c-c18fc264042c] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:42:41.411164 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-1e918dea-71a1-4977-9cb3-3058415f0325 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:41.411651 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-1e918dea-71a1-4977-9cb3-3058415f0325 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:41.412095 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-1e918dea-71a1-4977-9cb3-3058415f0325 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:41.412792 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-1e918dea-71a1-4977-9cb3-3058415f0325 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:41.413243 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-1e918dea-71a1-4977-9cb3-3058415f0325 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:41.413667 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-1e918dea-71a1-4977-9cb3-3058415f0325 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:41.553459 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-1e918dea-71a1-4977-9cb3-3058415f0325 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd3f0d615187d4779bdd7da8c9f24c796', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0151d58abd047a692f5cc4e7d05bea5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:42:41.561406 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-1e918dea-71a1-4977-9cb3-3058415f0325 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] 10.222.0.26 "GET /compute/v2.1/servers/3da61834-fd67-49d2-925c-c18fc264042c" status: 200 len: 1509 microversion: 2.1 time: 0.161117 Oct 19 00:42:41.561973 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 121/236] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:42:41 2022] GET /compute/v2.1/servers/3da61834-fd67-49d2-925c-c18fc264042c => generated 1509 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 19 00:42:41.590936 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-9229703d-50e1-4e10-a24f-7fcaa6606f45 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:42:41.591548 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-9229703d-50e1-4e10-a24f-7fcaa6606f45 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] [instance: c5d02d64-d4e6-445d-a155-b875b8a8cce0] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:42:41.599843 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-9229703d-50e1-4e10-a24f-7fcaa6606f45 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:41.600356 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-9229703d-50e1-4e10-a24f-7fcaa6606f45 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:41.600791 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-9229703d-50e1-4e10-a24f-7fcaa6606f45 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:41.601486 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-9229703d-50e1-4e10-a24f-7fcaa6606f45 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:41.602758 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-9229703d-50e1-4e10-a24f-7fcaa6606f45 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:41.603216 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-9229703d-50e1-4e10-a24f-7fcaa6606f45 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:41.738211 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-9229703d-50e1-4e10-a24f-7fcaa6606f45 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a08c9457231a447eb1c4130e68cfb31d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '173a52b79dab4962a4aa380c26f97d2d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:42:41.747529 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-9229703d-50e1-4e10-a24f-7fcaa6606f45 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] 10.222.0.26 "GET /compute/v2.1/servers/c5d02d64-d4e6-445d-a155-b875b8a8cce0" status: 200 len: 1314 microversion: 2.1 time: 0.160304 Oct 19 00:42:41.748249 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 122/237] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:42:41 2022] GET /compute/v2.1/servers/c5d02d64-d4e6-445d-a155-b875b8a8cce0 => generated 1314 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 19 00:42:42.081133 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-d8c83841-ceee-474d-a48e-9d5b1836e982 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:42:42.081768 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-d8c83841-ceee-474d-a48e-9d5b1836e982 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] [instance: b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:42:42.091306 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d8c83841-ceee-474d-a48e-9d5b1836e982 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:42.091791 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d8c83841-ceee-474d-a48e-9d5b1836e982 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:42.092832 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d8c83841-ceee-474d-a48e-9d5b1836e982 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:42.093682 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d8c83841-ceee-474d-a48e-9d5b1836e982 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:42.094133 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d8c83841-ceee-474d-a48e-9d5b1836e982 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:42.094577 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d8c83841-ceee-474d-a48e-9d5b1836e982 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:42.330896 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-d8c83841-ceee-474d-a48e-9d5b1836e982 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:42:42.341318 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-d8c83841-ceee-474d-a48e-9d5b1836e982 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:42:42.342897 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-d8c83841-ceee-474d-a48e-9d5b1836e982 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:42:42.344238 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-d8c83841-ceee-474d-a48e-9d5b1836e982 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] 10.222.0.26 "GET /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2" status: 200 len: 1723 microversion: 2.20 time: 0.266952 Oct 19 00:42:42.344798 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 123/238] 10.222.0.26 () {60 vars in 1277 bytes} [Wed Oct 19 00:42:42 2022] GET /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2 => generated 1723 bytes in 268 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 19 00:42:42.585255 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-53ceefe9-a7eb-4ac5-a821-f92f82b6874c tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:42:42.590887 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-53ceefe9-a7eb-4ac5-a821-f92f82b6874c tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] [instance: 3da61834-fd67-49d2-925c-c18fc264042c] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:42:42.599370 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-53ceefe9-a7eb-4ac5-a821-f92f82b6874c tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:42.600050 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-53ceefe9-a7eb-4ac5-a821-f92f82b6874c tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:42.600660 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-53ceefe9-a7eb-4ac5-a821-f92f82b6874c tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:42.601785 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-53ceefe9-a7eb-4ac5-a821-f92f82b6874c tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:42.602545 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-53ceefe9-a7eb-4ac5-a821-f92f82b6874c tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:42.603300 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-53ceefe9-a7eb-4ac5-a821-f92f82b6874c tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:43.011925 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-53ceefe9-a7eb-4ac5-a821-f92f82b6874c tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd3f0d615187d4779bdd7da8c9f24c796', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0151d58abd047a692f5cc4e7d05bea5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:42:43.022354 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-53ceefe9-a7eb-4ac5-a821-f92f82b6874c tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] 10.222.0.26 "GET /compute/v2.1/servers/3da61834-fd67-49d2-925c-c18fc264042c" status: 200 len: 1509 microversion: 2.1 time: 0.441133 Oct 19 00:42:43.034511 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 124/239] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:42:42 2022] GET /compute/v2.1/servers/3da61834-fd67-49d2-925c-c18fc264042c => generated 1509 bytes in 443 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:42:43.034511 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-965580ad-55d8-4d39-8605-79d649338ddb tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:42:43.034511 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-965580ad-55d8-4d39-8605-79d649338ddb tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] [instance: c5d02d64-d4e6-445d-a155-b875b8a8cce0] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:42:43.040705 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-965580ad-55d8-4d39-8605-79d649338ddb tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:43.040705 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-965580ad-55d8-4d39-8605-79d649338ddb tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:43.040705 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-965580ad-55d8-4d39-8605-79d649338ddb tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:43.040705 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-965580ad-55d8-4d39-8605-79d649338ddb tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:43.041098 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-965580ad-55d8-4d39-8605-79d649338ddb tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:43.041098 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-965580ad-55d8-4d39-8605-79d649338ddb tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:43.191581 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-965580ad-55d8-4d39-8605-79d649338ddb tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a08c9457231a447eb1c4130e68cfb31d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '173a52b79dab4962a4aa380c26f97d2d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:42:43.200688 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-965580ad-55d8-4d39-8605-79d649338ddb tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] 10.222.0.26 "GET /compute/v2.1/servers/c5d02d64-d4e6-445d-a155-b875b8a8cce0" status: 200 len: 1314 microversion: 2.1 time: 0.176474 Oct 19 00:42:43.202252 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 125/240] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:42:43 2022] GET /compute/v2.1/servers/c5d02d64-d4e6-445d-a155-b875b8a8cce0 => generated 1314 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:42:43.229409 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-5964f6b6-841b-4a94-a15f-7799ee21062d tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Action: 'create', calling method: .version_select at 0x7f71c3fd9790>, body: {"keypair": {"name": "tempest-keypair-356906562", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBGyUVUr4GV952F3qsBDp1Gpqt55xAxEkiVNK/PTG3vulSzUyrkBdLS/MiExJoqoq+KHQRM+9lIwc06WKatpayZJhLJNyzkFSbbLP+tDDFZ0txuCXJFCAv8nrN5odiNYCaQ=="}} {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 19 00:42:43.251472 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.quota [None req-5964f6b6-841b-4a94-a15f-7799ee21062d tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Getting quotas for project 0b35c2038f6b4d0a9012b004f1416c00. Resources: {'key_pairs'} {{(pid=125641) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 19 00:42:43.255723 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.quota [None req-5964f6b6-841b-4a94-a15f-7799ee21062d tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Getting quotas for user 6f8725c4d05b4dd79b05b43fba15980b and project 0b35c2038f6b4d0a9012b004f1416c00. Resources: {'key_pairs'} {{(pid=125641) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 19 00:42:43.296101 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-5964f6b6-841b-4a94-a15f-7799ee21062d tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] 10.222.0.26 "POST /compute/v2.1/os-keypairs" status: 200 len: 385 microversion: 2.1 time: 0.070272 Oct 19 00:42:43.296853 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 126/241] 10.222.0.26 () {60 vars in 1159 bytes} [Wed Oct 19 00:42:43 2022] POST /compute/v2.1/os-keypairs => generated 385 bytes in 72 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 19 00:42:43.374005 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-9dc95b21-21dc-440b-8d60-74e14d5836a4 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:42:43.374669 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-9dc95b21-21dc-440b-8d60-74e14d5836a4 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] [instance: b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:42:43.376816 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG dbcounter [-] [125641] Writing DB stats nova_cell0:SELECT=2 {{(pid=125641) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 00:42:43.384542 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-9dc95b21-21dc-440b-8d60-74e14d5836a4 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:43.385182 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-9dc95b21-21dc-440b-8d60-74e14d5836a4 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:43.385670 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-9dc95b21-21dc-440b-8d60-74e14d5836a4 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:43.386762 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-9dc95b21-21dc-440b-8d60-74e14d5836a4 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:43.387353 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-9dc95b21-21dc-440b-8d60-74e14d5836a4 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:43.387792 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-9dc95b21-21dc-440b-8d60-74e14d5836a4 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:43.543103 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-9dc95b21-21dc-440b-8d60-74e14d5836a4 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:42:43.554288 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-9dc95b21-21dc-440b-8d60-74e14d5836a4 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:42:43.556280 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-9dc95b21-21dc-440b-8d60-74e14d5836a4 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:42:43.556948 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-9dc95b21-21dc-440b-8d60-74e14d5836a4 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] 10.222.0.26 "GET /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2" status: 200 len: 1723 microversion: 2.20 time: 0.188009 Oct 19 00:42:43.557731 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 127/242] 10.222.0.26 () {60 vars in 1277 bytes} [Wed Oct 19 00:42:43 2022] GET /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2 => generated 1723 bytes in 189 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 19 00:42:44.050310 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-ca9bc372-66a6-4dae-a060-cca2c4acc9ff tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:42:44.051108 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-ca9bc372-66a6-4dae-a060-cca2c4acc9ff tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] [instance: 3da61834-fd67-49d2-925c-c18fc264042c] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:42:44.059202 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ca9bc372-66a6-4dae-a060-cca2c4acc9ff tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:44.060635 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ca9bc372-66a6-4dae-a060-cca2c4acc9ff tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:44.061397 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ca9bc372-66a6-4dae-a060-cca2c4acc9ff tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:44.062476 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ca9bc372-66a6-4dae-a060-cca2c4acc9ff tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:44.063165 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ca9bc372-66a6-4dae-a060-cca2c4acc9ff tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:44.063785 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ca9bc372-66a6-4dae-a060-cca2c4acc9ff tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:44.368581 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-ca9bc372-66a6-4dae-a060-cca2c4acc9ff tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd3f0d615187d4779bdd7da8c9f24c796', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0151d58abd047a692f5cc4e7d05bea5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:42:44.378172 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-ca9bc372-66a6-4dae-a060-cca2c4acc9ff tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] 10.222.0.26 "GET /compute/v2.1/servers/3da61834-fd67-49d2-925c-c18fc264042c" status: 200 len: 1509 microversion: 2.1 time: 0.333244 Oct 19 00:42:44.378459 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 128/243] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:42:44 2022] GET /compute/v2.1/servers/3da61834-fd67-49d2-925c-c18fc264042c => generated 1509 bytes in 334 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 19 00:42:44.383376 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-b5bb770d-27c8-4a4f-a019-42edef9bf662 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:42:44.384400 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-b5bb770d-27c8-4a4f-a019-42edef9bf662 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] [instance: c5d02d64-d4e6-445d-a155-b875b8a8cce0] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:42:44.391441 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-b5bb770d-27c8-4a4f-a019-42edef9bf662 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:44.392131 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-b5bb770d-27c8-4a4f-a019-42edef9bf662 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:44.392713 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-b5bb770d-27c8-4a4f-a019-42edef9bf662 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:44.393583 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-b5bb770d-27c8-4a4f-a019-42edef9bf662 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:44.394182 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-b5bb770d-27c8-4a4f-a019-42edef9bf662 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:44.395005 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-b5bb770d-27c8-4a4f-a019-42edef9bf662 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:44.619444 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-b5bb770d-27c8-4a4f-a019-42edef9bf662 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a08c9457231a447eb1c4130e68cfb31d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '173a52b79dab4962a4aa380c26f97d2d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:42:44.627958 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-b5bb770d-27c8-4a4f-a019-42edef9bf662 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] 10.222.0.26 "GET /compute/v2.1/servers/c5d02d64-d4e6-445d-a155-b875b8a8cce0" status: 200 len: 1314 microversion: 2.1 time: 0.248259 Oct 19 00:42:44.628651 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 129/244] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:42:44 2022] GET /compute/v2.1/servers/c5d02d64-d4e6-445d-a155-b875b8a8cce0 => generated 1314 bytes in 250 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:42:44.633725 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-25c4e97f-59b5-40b0-b14b-bb4517a97688 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:42:44.634415 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-25c4e97f-59b5-40b0-b14b-bb4517a97688 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] [instance: b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:42:44.641137 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-25c4e97f-59b5-40b0-b14b-bb4517a97688 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:44.641805 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-25c4e97f-59b5-40b0-b14b-bb4517a97688 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:44.642318 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-25c4e97f-59b5-40b0-b14b-bb4517a97688 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:44.643225 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-25c4e97f-59b5-40b0-b14b-bb4517a97688 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:44.643893 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-25c4e97f-59b5-40b0-b14b-bb4517a97688 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:44.644391 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-25c4e97f-59b5-40b0-b14b-bb4517a97688 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:44.864081 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-25c4e97f-59b5-40b0-b14b-bb4517a97688 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:42:44.874723 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-25c4e97f-59b5-40b0-b14b-bb4517a97688 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:42:44.877428 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-25c4e97f-59b5-40b0-b14b-bb4517a97688 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:42:44.879942 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-25c4e97f-59b5-40b0-b14b-bb4517a97688 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] 10.222.0.26 "GET /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2" status: 200 len: 1723 microversion: 2.20 time: 0.249860 Oct 19 00:42:44.881171 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 130/245] 10.222.0.26 () {60 vars in 1277 bytes} [Wed Oct 19 00:42:44 2022] GET /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2 => generated 1723 bytes in 252 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 19 00:42:44.954740 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-d96172c3-00b1-4877-aff2-9c44f51edc23 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeNegativeTest-server-1254132818", "imageRef": "b387b4f8-f75e-4a68-b49c-a015bf61c46b", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "635fade4-7ee9-45f4-8020-ceb7d6bf784a"}], "security_groups": [{"name": "tempest-securitygroup--1258616963"}], "key_name": "tempest-keypair-356906562", "user_data": "}} {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 19 00:42:45.067059 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.network.neutron [None req-d96172c3-00b1-4877-aff2-9c44f51edc23 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] validate_networks() for [('635fade4-7ee9-45f4-8020-ceb7d6bf784a', None, None, None, None, None)] {{(pid=125641) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 19 00:42:45.493644 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.virt.hardware [None req-d96172c3-00b1-4877-aff2-9c44f51edc23 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Flavor limits 0:0:0 {{(pid=125641) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 19 00:42:45.494212 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.virt.hardware [None req-d96172c3-00b1-4877-aff2-9c44f51edc23 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Image limits 0:0:0 {{(pid=125641) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 19 00:42:45.494568 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.virt.hardware [None req-d96172c3-00b1-4877-aff2-9c44f51edc23 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Flavor pref 0:0:0 {{(pid=125641) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 19 00:42:45.494988 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.virt.hardware [None req-d96172c3-00b1-4877-aff2-9c44f51edc23 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Image pref 0:0:0 {{(pid=125641) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 19 00:42:45.495405 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.virt.hardware [None req-d96172c3-00b1-4877-aff2-9c44f51edc23 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=125641) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 19 00:42:45.505682 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d96172c3-00b1-4877-aff2-9c44f51edc23 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:45.506174 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d96172c3-00b1-4877-aff2-9c44f51edc23 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:45.506656 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d96172c3-00b1-4877-aff2-9c44f51edc23 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:45.507387 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d96172c3-00b1-4877-aff2-9c44f51edc23 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:45.507845 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d96172c3-00b1-4877-aff2-9c44f51edc23 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:45.508274 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d96172c3-00b1-4877-aff2-9c44f51edc23 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:45.526046 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.quota [None req-d96172c3-00b1-4877-aff2-9c44f51edc23 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Getting quotas for project 0b35c2038f6b4d0a9012b004f1416c00. Resources: {'cores', 'ram', 'instances'} {{(pid=125641) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 19 00:42:45.530563 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.quota [None req-d96172c3-00b1-4877-aff2-9c44f51edc23 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Getting quotas for user 6f8725c4d05b4dd79b05b43fba15980b and project 0b35c2038f6b4d0a9012b004f1416c00. Resources: {'cores', 'ram', 'instances'} {{(pid=125641) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 19 00:42:45.541738 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-d96172c3-00b1-4877-aff2-9c44f51edc23 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Going to run 1 instances... {{(pid=125641) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 19 00:42:45.544098 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-d96172c3-00b1-4877-aff2-9c44f51edc23 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] [instance: 25d64f9f-18a6-4b6a-92b1-3e8e7d017ab8] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='b387b4f8-f75e-4a68-b49c-a015bf61c46b',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=125641) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 19 00:42:45.556996 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG dbcounter [-] [125641] Writing DB stats nova_api:SELECT=95,nova_api:INSERT=5 {{(pid=125641) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 00:42:45.597802 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-d96172c3-00b1-4877-aff2-9c44f51edc23 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 202 len: 402 microversion: 2.1 time: 0.647419 Oct 19 00:42:45.598539 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 131/246] 10.222.0.26 () {60 vars in 1147 bytes} [Wed Oct 19 00:42:44 2022] POST /compute/v2.1/servers => generated 402 bytes in 649 msecs (HTTP/1.1 202) 10 headers in 451 bytes (2 switches on core 0) Oct 19 00:42:45.606072 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-f699fc9e-7d02-41df-b7b8-fcd64d95e2dc tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:42:45.606072 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-f699fc9e-7d02-41df-b7b8-fcd64d95e2dc tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] [instance: 3da61834-fd67-49d2-925c-c18fc264042c] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:42:45.611161 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-f699fc9e-7d02-41df-b7b8-fcd64d95e2dc tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:45.611652 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-f699fc9e-7d02-41df-b7b8-fcd64d95e2dc tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:45.612102 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-f699fc9e-7d02-41df-b7b8-fcd64d95e2dc tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:45.612876 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-f699fc9e-7d02-41df-b7b8-fcd64d95e2dc tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:45.613333 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-f699fc9e-7d02-41df-b7b8-fcd64d95e2dc tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:45.613751 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-f699fc9e-7d02-41df-b7b8-fcd64d95e2dc tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:45.786705 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-f699fc9e-7d02-41df-b7b8-fcd64d95e2dc tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd3f0d615187d4779bdd7da8c9f24c796', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0151d58abd047a692f5cc4e7d05bea5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:42:45.796755 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-f699fc9e-7d02-41df-b7b8-fcd64d95e2dc tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] 10.222.0.26 "GET /compute/v2.1/servers/3da61834-fd67-49d2-925c-c18fc264042c" status: 200 len: 1509 microversion: 2.1 time: 0.197098 Oct 19 00:42:45.797303 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 132/247] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:42:45 2022] GET /compute/v2.1/servers/3da61834-fd67-49d2-925c-c18fc264042c => generated 1509 bytes in 198 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:42:45.801995 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-4d851da9-3a44-4c25-9800-23d8d91d84b8 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:42:45.802630 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-4d851da9-3a44-4c25-9800-23d8d91d84b8 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] [instance: 25d64f9f-18a6-4b6a-92b1-3e8e7d017ab8] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:42:45.856845 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-4d851da9-3a44-4c25-9800-23d8d91d84b8 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f8725c4d05b4dd79b05b43fba15980b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0b35c2038f6b4d0a9012b004f1416c00', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:42:45.867871 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-4d851da9-3a44-4c25-9800-23d8d91d84b8 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] 10.222.0.26 "GET /compute/v2.1/servers/25d64f9f-18a6-4b6a-92b1-3e8e7d017ab8" status: 200 len: 1241 microversion: 2.1 time: 0.069045 Oct 19 00:42:45.868806 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 133/248] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:42:45 2022] GET /compute/v2.1/servers/25d64f9f-18a6-4b6a-92b1-3e8e7d017ab8 => generated 1241 bytes in 71 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:42:45.874661 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-d0c2b068-45b5-402b-809c-6c83832e496e tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:42:45.875487 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-d0c2b068-45b5-402b-809c-6c83832e496e tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] [instance: c5d02d64-d4e6-445d-a155-b875b8a8cce0] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:42:45.887130 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d0c2b068-45b5-402b-809c-6c83832e496e tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:45.887130 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d0c2b068-45b5-402b-809c-6c83832e496e tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:45.887130 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d0c2b068-45b5-402b-809c-6c83832e496e tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:45.887130 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d0c2b068-45b5-402b-809c-6c83832e496e tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:45.887547 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d0c2b068-45b5-402b-809c-6c83832e496e tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:45.887547 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d0c2b068-45b5-402b-809c-6c83832e496e tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:46.040749 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-d0c2b068-45b5-402b-809c-6c83832e496e tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a08c9457231a447eb1c4130e68cfb31d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '173a52b79dab4962a4aa380c26f97d2d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:42:46.052265 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-d0c2b068-45b5-402b-809c-6c83832e496e tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] 10.222.0.26 "GET /compute/v2.1/servers/c5d02d64-d4e6-445d-a155-b875b8a8cce0" status: 200 len: 1314 microversion: 2.1 time: 0.181992 Oct 19 00:42:46.052945 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 134/249] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:42:45 2022] GET /compute/v2.1/servers/c5d02d64-d4e6-445d-a155-b875b8a8cce0 => generated 1314 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:42:46.057663 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-689196ce-b82a-4b3f-9299-ec68af74efea tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:42:46.058275 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-689196ce-b82a-4b3f-9299-ec68af74efea tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] [instance: b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:42:46.064823 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-689196ce-b82a-4b3f-9299-ec68af74efea tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:46.065306 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-689196ce-b82a-4b3f-9299-ec68af74efea tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:46.065920 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-689196ce-b82a-4b3f-9299-ec68af74efea tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:46.066719 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-689196ce-b82a-4b3f-9299-ec68af74efea tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:46.067220 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-689196ce-b82a-4b3f-9299-ec68af74efea tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:46.067595 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-689196ce-b82a-4b3f-9299-ec68af74efea tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:46.223527 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-689196ce-b82a-4b3f-9299-ec68af74efea tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:42:46.232643 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-689196ce-b82a-4b3f-9299-ec68af74efea tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:42:46.234184 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-689196ce-b82a-4b3f-9299-ec68af74efea tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:42:46.235387 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-689196ce-b82a-4b3f-9299-ec68af74efea tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] 10.222.0.26 "GET /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2" status: 200 len: 1723 microversion: 2.20 time: 0.181514 Oct 19 00:42:46.235957 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 135/250] 10.222.0.26 () {60 vars in 1277 bytes} [Wed Oct 19 00:42:46 2022] GET /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2 => generated 1723 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 19 00:42:46.821514 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-cb8f95ef-910f-4789-af0c-6038db4ff2d1 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:42:46.822881 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-cb8f95ef-910f-4789-af0c-6038db4ff2d1 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] [instance: 3da61834-fd67-49d2-925c-c18fc264042c] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:42:46.829847 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-cb8f95ef-910f-4789-af0c-6038db4ff2d1 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:46.830781 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-cb8f95ef-910f-4789-af0c-6038db4ff2d1 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:46.831401 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-cb8f95ef-910f-4789-af0c-6038db4ff2d1 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:46.832274 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-cb8f95ef-910f-4789-af0c-6038db4ff2d1 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:46.832892 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-cb8f95ef-910f-4789-af0c-6038db4ff2d1 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:46.833488 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-cb8f95ef-910f-4789-af0c-6038db4ff2d1 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:47.038690 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-cb8f95ef-910f-4789-af0c-6038db4ff2d1 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd3f0d615187d4779bdd7da8c9f24c796', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0151d58abd047a692f5cc4e7d05bea5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:42:47.049376 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-cb8f95ef-910f-4789-af0c-6038db4ff2d1 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] 10.222.0.26 "GET /compute/v2.1/servers/3da61834-fd67-49d2-925c-c18fc264042c" status: 200 len: 1515 microversion: 2.1 time: 0.231983 Oct 19 00:42:47.050699 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 136/251] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:42:46 2022] GET /compute/v2.1/servers/3da61834-fd67-49d2-925c-c18fc264042c => generated 1515 bytes in 234 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:42:47.087751 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-f00988d5-7daf-4e12-a183-b27a684ca8f5 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:42:47.090564 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-f00988d5-7daf-4e12-a183-b27a684ca8f5 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] [instance: 25d64f9f-18a6-4b6a-92b1-3e8e7d017ab8] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:42:47.142056 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-f00988d5-7daf-4e12-a183-b27a684ca8f5 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:47.143375 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-f00988d5-7daf-4e12-a183-b27a684ca8f5 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:47.144076 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-f00988d5-7daf-4e12-a183-b27a684ca8f5 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:47.145288 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-f00988d5-7daf-4e12-a183-b27a684ca8f5 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:47.146536 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-f00988d5-7daf-4e12-a183-b27a684ca8f5 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:47.147676 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-f00988d5-7daf-4e12-a183-b27a684ca8f5 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:47.296801 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-f00988d5-7daf-4e12-a183-b27a684ca8f5 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f8725c4d05b4dd79b05b43fba15980b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0b35c2038f6b4d0a9012b004f1416c00', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:42:47.323873 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-f00988d5-7daf-4e12-a183-b27a684ca8f5 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] 10.222.0.26 "GET /compute/v2.1/servers/25d64f9f-18a6-4b6a-92b1-3e8e7d017ab8" status: 200 len: 1293 microversion: 2.1 time: 0.271472 Oct 19 00:42:47.324951 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 137/252] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:42:47 2022] GET /compute/v2.1/servers/25d64f9f-18a6-4b6a-92b1-3e8e7d017ab8 => generated 1293 bytes in 273 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:42:47.350340 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-8c675e25-9caa-4ac3-ab11-ff01982df4da tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:42:47.351310 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-8c675e25-9caa-4ac3-ab11-ff01982df4da tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] [instance: c5d02d64-d4e6-445d-a155-b875b8a8cce0] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:42:47.357976 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-8c675e25-9caa-4ac3-ab11-ff01982df4da tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:47.358785 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-8c675e25-9caa-4ac3-ab11-ff01982df4da tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:47.359789 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-8c675e25-9caa-4ac3-ab11-ff01982df4da tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:47.360977 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-8c675e25-9caa-4ac3-ab11-ff01982df4da tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:47.361855 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-8c675e25-9caa-4ac3-ab11-ff01982df4da tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:47.362580 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-8c675e25-9caa-4ac3-ab11-ff01982df4da tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:47.392820 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG dbcounter [-] [125641] Writing DB stats nova_cell1:SELECT=100 {{(pid=125641) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 00:42:47.461529 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG dbcounter [-] [125640] Writing DB stats nova_cell1:SELECT=50,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=125640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 00:42:47.630389 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-8c675e25-9caa-4ac3-ab11-ff01982df4da tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a08c9457231a447eb1c4130e68cfb31d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '173a52b79dab4962a4aa380c26f97d2d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:42:47.638212 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-8c675e25-9caa-4ac3-ab11-ff01982df4da tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] 10.222.0.26 "GET /compute/v2.1/servers/c5d02d64-d4e6-445d-a155-b875b8a8cce0" status: 200 len: 1314 microversion: 2.1 time: 0.297608 Oct 19 00:42:47.656053 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 138/253] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:42:47 2022] GET /compute/v2.1/servers/c5d02d64-d4e6-445d-a155-b875b8a8cce0 => generated 1314 bytes in 316 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:42:47.665060 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-b16d34e2-9236-40a3-afe6-f056ae6ffe96 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:42:47.666028 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: DEBUG dbcounter [-] [125640] Writing DB stats nova_api:SELECT=5 {{(pid=125640) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 00:42:47.670748 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-b16d34e2-9236-40a3-afe6-f056ae6ffe96 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] [instance: b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:42:47.698357 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-b16d34e2-9236-40a3-afe6-f056ae6ffe96 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:47.701044 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-b16d34e2-9236-40a3-afe6-f056ae6ffe96 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:47.705346 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-b16d34e2-9236-40a3-afe6-f056ae6ffe96 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:47.709985 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-b16d34e2-9236-40a3-afe6-f056ae6ffe96 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:47.710956 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-b16d34e2-9236-40a3-afe6-f056ae6ffe96 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:47.711797 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-b16d34e2-9236-40a3-afe6-f056ae6ffe96 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:47.895103 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-b16d34e2-9236-40a3-afe6-f056ae6ffe96 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:42:47.906282 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-b16d34e2-9236-40a3-afe6-f056ae6ffe96 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:42:47.908359 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-b16d34e2-9236-40a3-afe6-f056ae6ffe96 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:42:47.909563 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-b16d34e2-9236-40a3-afe6-f056ae6ffe96 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] 10.222.0.26 "GET /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2" status: 200 len: 1723 microversion: 2.20 time: 0.252175 Oct 19 00:42:47.910121 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 139/254] 10.222.0.26 () {60 vars in 1277 bytes} [Wed Oct 19 00:42:47 2022] GET /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2 => generated 1723 bytes in 253 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 19 00:42:48.353617 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-54fac97f-4362-4fbd-8631-3d6ce495ed2c tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:42:48.354919 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-54fac97f-4362-4fbd-8631-3d6ce495ed2c tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] [instance: 25d64f9f-18a6-4b6a-92b1-3e8e7d017ab8] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:42:48.363295 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-54fac97f-4362-4fbd-8631-3d6ce495ed2c tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:48.365100 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-54fac97f-4362-4fbd-8631-3d6ce495ed2c tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:48.365756 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-54fac97f-4362-4fbd-8631-3d6ce495ed2c tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:48.366996 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-54fac97f-4362-4fbd-8631-3d6ce495ed2c tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:48.368224 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-54fac97f-4362-4fbd-8631-3d6ce495ed2c tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:48.370099 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-54fac97f-4362-4fbd-8631-3d6ce495ed2c tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:48.704433 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-54fac97f-4362-4fbd-8631-3d6ce495ed2c tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f8725c4d05b4dd79b05b43fba15980b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0b35c2038f6b4d0a9012b004f1416c00', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:42:48.725674 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-54fac97f-4362-4fbd-8631-3d6ce495ed2c tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] 10.222.0.26 "GET /compute/v2.1/servers/25d64f9f-18a6-4b6a-92b1-3e8e7d017ab8" status: 200 len: 1367 microversion: 2.1 time: 0.375785 Oct 19 00:42:48.728118 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 140/255] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:42:48 2022] GET /compute/v2.1/servers/25d64f9f-18a6-4b6a-92b1-3e8e7d017ab8 => generated 1367 bytes in 380 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 19 00:42:48.741671 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-89269010-9ca3-49c5-a8e8-a156f415aabe tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:42:48.745352 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-89269010-9ca3-49c5-a8e8-a156f415aabe tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] [instance: c5d02d64-d4e6-445d-a155-b875b8a8cce0] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:42:48.775460 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-89269010-9ca3-49c5-a8e8-a156f415aabe tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:48.779804 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-89269010-9ca3-49c5-a8e8-a156f415aabe tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:48.782946 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-89269010-9ca3-49c5-a8e8-a156f415aabe tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:48.786264 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-89269010-9ca3-49c5-a8e8-a156f415aabe tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:48.787674 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-89269010-9ca3-49c5-a8e8-a156f415aabe tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:48.789236 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-89269010-9ca3-49c5-a8e8-a156f415aabe tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:49.009451 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-89269010-9ca3-49c5-a8e8-a156f415aabe tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a08c9457231a447eb1c4130e68cfb31d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '173a52b79dab4962a4aa380c26f97d2d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:42:49.029395 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-89269010-9ca3-49c5-a8e8-a156f415aabe tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] 10.222.0.26 "GET /compute/v2.1/servers/c5d02d64-d4e6-445d-a155-b875b8a8cce0" status: 200 len: 1314 microversion: 2.1 time: 0.293247 Oct 19 00:42:49.030464 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 141/256] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:42:48 2022] GET /compute/v2.1/servers/c5d02d64-d4e6-445d-a155-b875b8a8cce0 => generated 1314 bytes in 302 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:42:49.037472 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-67dda5c0-0d61-4df1-b06e-3176fd598141 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Action: 'create', calling method: .version_select at 0x7f71c3fd9310>, body: {"keypair": {"name": "tempest-keypair-2094775421", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBOUycuG+BRsofwOewAHAgFh5wUk+1tv1ONEBZt8lMzB6dcEzzfOmjtuP0aYPjoSyyw6HEMKsEOZlo9RA+1jpJu3AhneYPOzWdaOZQjZXox5MrfdaEL+e5BSzHhnqj56rKA=="}} {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 19 00:42:49.060255 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.quota [None req-67dda5c0-0d61-4df1-b06e-3176fd598141 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Getting quotas for project c0151d58abd047a692f5cc4e7d05bea5. Resources: {'key_pairs'} {{(pid=125641) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 19 00:42:49.066231 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.quota [None req-67dda5c0-0d61-4df1-b06e-3176fd598141 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Getting quotas for user d3f0d615187d4779bdd7da8c9f24c796 and project c0151d58abd047a692f5cc4e7d05bea5. Resources: {'key_pairs'} {{(pid=125641) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 19 00:42:49.096231 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-67dda5c0-0d61-4df1-b06e-3176fd598141 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] 10.222.0.26 "POST /compute/v2.1/os-keypairs" status: 200 len: 386 microversion: 2.1 time: 0.063056 Oct 19 00:42:49.096956 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 142/257] 10.222.0.26 () {60 vars in 1159 bytes} [Wed Oct 19 00:42:49 2022] POST /compute/v2.1/os-keypairs => generated 386 bytes in 66 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 19 00:42:49.103308 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [req-a29fa6b4-12e5-45fc-afe5-45151355517e req-a160895f-3b22-4f2f-87f8-cebe915e3b9d service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "c5d02d64-d4e6-445d-a155-b875b8a8cce0", "name": "network-vif-plugged", "status": "completed", "tag": "1b4d823f-e9a3-4a43-a759-c283b8c3dac4"}]} {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 19 00:42:49.117152 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-a29fa6b4-12e5-45fc-afe5-45151355517e req-a160895f-3b22-4f2f-87f8-cebe915e3b9d service nova] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:49.117907 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-a29fa6b4-12e5-45fc-afe5-45151355517e req-a160895f-3b22-4f2f-87f8-cebe915e3b9d service nova] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:49.119871 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-a29fa6b4-12e5-45fc-afe5-45151355517e req-a160895f-3b22-4f2f-87f8-cebe915e3b9d service nova] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:49.135244 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.compute.server_external_events [req-a29fa6b4-12e5-45fc-afe5-45151355517e req-a160895f-3b22-4f2f-87f8-cebe915e3b9d service nova] Creating event network-vif-plugged:1b4d823f-e9a3-4a43-a759-c283b8c3dac4 for instance c5d02d64-d4e6-445d-a155-b875b8a8cce0 on hv-cinder-maste Oct 19 00:42:49.145213 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [req-a29fa6b4-12e5-45fc-afe5-45151355517e req-a160895f-3b22-4f2f-87f8-cebe915e3b9d service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.045767 Oct 19 00:42:49.146165 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 143/258] 10.222.0.26 () {64 vars in 1317 bytes} [Wed Oct 19 00:42:49 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 19 00:42:49.153711 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-34fcccd7-3551-45ec-9347-2608984195db tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:42:49.155201 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-34fcccd7-3551-45ec-9347-2608984195db tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] [instance: b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:42:49.165303 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-34fcccd7-3551-45ec-9347-2608984195db tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:49.170693 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-34fcccd7-3551-45ec-9347-2608984195db tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:49.170693 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-34fcccd7-3551-45ec-9347-2608984195db tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:49.170693 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-34fcccd7-3551-45ec-9347-2608984195db tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:49.170693 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-34fcccd7-3551-45ec-9347-2608984195db tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:49.171009 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-34fcccd7-3551-45ec-9347-2608984195db tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:49.888031 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-34fcccd7-3551-45ec-9347-2608984195db tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:42:49.897412 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-34fcccd7-3551-45ec-9347-2608984195db tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:42:49.899948 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-34fcccd7-3551-45ec-9347-2608984195db tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:42:49.901333 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-34fcccd7-3551-45ec-9347-2608984195db tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] 10.222.0.26 "GET /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2" status: 200 len: 1723 microversion: 2.20 time: 0.753684 Oct 19 00:42:49.913136 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 144/259] 10.222.0.26 () {60 vars in 1277 bytes} [Wed Oct 19 00:42:49 2022] GET /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2 => generated 1723 bytes in 765 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 19 00:42:49.923938 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-f15d96ad-0c5f-45b9-be0b-c6394db1322d tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:42:49.925053 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-f15d96ad-0c5f-45b9-be0b-c6394db1322d tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] [instance: 25d64f9f-18a6-4b6a-92b1-3e8e7d017ab8] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:42:49.943053 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-f15d96ad-0c5f-45b9-be0b-c6394db1322d tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:49.946527 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-f15d96ad-0c5f-45b9-be0b-c6394db1322d tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:49.947833 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-f15d96ad-0c5f-45b9-be0b-c6394db1322d tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:49.949644 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-f15d96ad-0c5f-45b9-be0b-c6394db1322d tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:49.953735 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-f15d96ad-0c5f-45b9-be0b-c6394db1322d tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:49.955192 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-f15d96ad-0c5f-45b9-be0b-c6394db1322d tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:50.277820 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-f15d96ad-0c5f-45b9-be0b-c6394db1322d tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f8725c4d05b4dd79b05b43fba15980b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0b35c2038f6b4d0a9012b004f1416c00', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:42:50.296799 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-f15d96ad-0c5f-45b9-be0b-c6394db1322d tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] 10.222.0.26 "GET /compute/v2.1/servers/25d64f9f-18a6-4b6a-92b1-3e8e7d017ab8" status: 200 len: 1367 microversion: 2.1 time: 0.381680 Oct 19 00:42:50.313992 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 145/260] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:42:49 2022] GET /compute/v2.1/servers/25d64f9f-18a6-4b6a-92b1-3e8e7d017ab8 => generated 1367 bytes in 396 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:42:50.320265 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [req-6f1c1516-8506-4469-b680-bf2b4b219c26 req-179255ab-5bcb-4dbd-979f-8f36ad54203a service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "25d64f9f-18a6-4b6a-92b1-3e8e7d017ab8", "tag": "720bfaf8-2ab1-4a23-bccc-396cef466b3a"}]} {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 19 00:42:50.332985 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-6f1c1516-8506-4469-b680-bf2b4b219c26 req-179255ab-5bcb-4dbd-979f-8f36ad54203a service nova] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:50.333402 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-6f1c1516-8506-4469-b680-bf2b4b219c26 req-179255ab-5bcb-4dbd-979f-8f36ad54203a service nova] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:50.334010 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-6f1c1516-8506-4469-b680-bf2b4b219c26 req-179255ab-5bcb-4dbd-979f-8f36ad54203a service nova] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:50.346901 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.compute.server_external_events [req-6f1c1516-8506-4469-b680-bf2b4b219c26 req-179255ab-5bcb-4dbd-979f-8f36ad54203a service nova] Creating event network-changed:720bfaf8-2ab1-4a23-bccc-396cef466b3a for instance 25d64f9f-18a6-4b6a-92b1-3e8e7d017ab8 on hv-cinder-maste Oct 19 00:42:50.352696 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [req-6f1c1516-8506-4469-b680-bf2b4b219c26 req-179255ab-5bcb-4dbd-979f-8f36ad54203a service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.036694 Oct 19 00:42:50.353878 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 146/261] 10.222.0.26 () {64 vars in 1317 bytes} [Wed Oct 19 00:42:50 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 19 00:42:50.360177 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-1ff00e1f-a5f0-4c1a-b1f3-08a54b6571a1 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:42:50.361400 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-1ff00e1f-a5f0-4c1a-b1f3-08a54b6571a1 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] [instance: c5d02d64-d4e6-445d-a155-b875b8a8cce0] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:42:50.368955 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-1ff00e1f-a5f0-4c1a-b1f3-08a54b6571a1 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:50.370022 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-1ff00e1f-a5f0-4c1a-b1f3-08a54b6571a1 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:50.370654 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-1ff00e1f-a5f0-4c1a-b1f3-08a54b6571a1 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:50.372589 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-1ff00e1f-a5f0-4c1a-b1f3-08a54b6571a1 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:50.373760 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-1ff00e1f-a5f0-4c1a-b1f3-08a54b6571a1 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:50.374348 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-1ff00e1f-a5f0-4c1a-b1f3-08a54b6571a1 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:50.688406 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-1ff00e1f-a5f0-4c1a-b1f3-08a54b6571a1 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a08c9457231a447eb1c4130e68cfb31d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '173a52b79dab4962a4aa380c26f97d2d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:42:50.702790 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-1ff00e1f-a5f0-4c1a-b1f3-08a54b6571a1 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] 10.222.0.26 "GET /compute/v2.1/servers/c5d02d64-d4e6-445d-a155-b875b8a8cce0" status: 200 len: 1314 microversion: 2.1 time: 0.346909 Oct 19 00:42:50.703490 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 147/262] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:42:50 2022] GET /compute/v2.1/servers/c5d02d64-d4e6-445d-a155-b875b8a8cce0 => generated 1314 bytes in 349 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:42:50.943599 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-62963644-d812-4022-afb4-c02ad7501850 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:42:50.945452 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-62963644-d812-4022-afb4-c02ad7501850 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] [instance: b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:42:50.961206 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-62963644-d812-4022-afb4-c02ad7501850 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:50.962652 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-62963644-d812-4022-afb4-c02ad7501850 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:50.963821 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-62963644-d812-4022-afb4-c02ad7501850 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:50.967431 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-62963644-d812-4022-afb4-c02ad7501850 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:50.968570 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-62963644-d812-4022-afb4-c02ad7501850 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:50.970788 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-62963644-d812-4022-afb4-c02ad7501850 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:51.234549 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-62963644-d812-4022-afb4-c02ad7501850 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:42:51.245466 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-62963644-d812-4022-afb4-c02ad7501850 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:42:51.247999 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-62963644-d812-4022-afb4-c02ad7501850 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:42:51.250351 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-62963644-d812-4022-afb4-c02ad7501850 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] 10.222.0.26 "GET /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2" status: 200 len: 1723 microversion: 2.20 time: 0.324684 Oct 19 00:42:51.251280 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 148/263] 10.222.0.26 () {60 vars in 1277 bytes} [Wed Oct 19 00:42:50 2022] GET /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2 => generated 1723 bytes in 327 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 19 00:42:51.328867 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-b3f8a870-ef40-486f-b84e-3c601b591617 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:42:51.330920 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-b3f8a870-ef40-486f-b84e-3c601b591617 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] [instance: 25d64f9f-18a6-4b6a-92b1-3e8e7d017ab8] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:42:51.339472 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-b3f8a870-ef40-486f-b84e-3c601b591617 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:51.340254 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-b3f8a870-ef40-486f-b84e-3c601b591617 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:51.340690 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-b3f8a870-ef40-486f-b84e-3c601b591617 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:51.341395 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-b3f8a870-ef40-486f-b84e-3c601b591617 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:51.341983 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-b3f8a870-ef40-486f-b84e-3c601b591617 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:51.342394 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-b3f8a870-ef40-486f-b84e-3c601b591617 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:51.537817 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-b3f8a870-ef40-486f-b84e-3c601b591617 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f8725c4d05b4dd79b05b43fba15980b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0b35c2038f6b4d0a9012b004f1416c00', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:42:51.565088 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-b3f8a870-ef40-486f-b84e-3c601b591617 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] 10.222.0.26 "GET /compute/v2.1/servers/25d64f9f-18a6-4b6a-92b1-3e8e7d017ab8" status: 200 len: 1367 microversion: 2.1 time: 0.244420 Oct 19 00:42:51.580189 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 149/264] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:42:51 2022] GET /compute/v2.1/servers/25d64f9f-18a6-4b6a-92b1-3e8e7d017ab8 => generated 1367 bytes in 263 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:42:51.694622 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-7009a74e-f8ba-425f-a9e4-0f2cc653ca9a tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerRescueNegativeTestJSON-server-2141811802", "imageRef": "b387b4f8-f75e-4a68-b49c-a015bf61c46b", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "75c84943-00ac-41e3-a6c3-2ea734b46148"}], "security_groups": [{"name": "tempest-securitygroup--1964336820"}], "key_name": "tempest-keypair-2094775421", "user_data": "}} {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 19 00:42:51.849446 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.network.neutron [None req-7009a74e-f8ba-425f-a9e4-0f2cc653ca9a tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] validate_networks() for [('75c84943-00ac-41e3-a6c3-2ea734b46148', None, None, None, None, None)] {{(pid=125641) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 19 00:42:52.598210 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.virt.hardware [None req-7009a74e-f8ba-425f-a9e4-0f2cc653ca9a tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Flavor limits 0:0:0 {{(pid=125641) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 19 00:42:52.599434 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.virt.hardware [None req-7009a74e-f8ba-425f-a9e4-0f2cc653ca9a tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Image limits 0:0:0 {{(pid=125641) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 19 00:42:52.599434 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.virt.hardware [None req-7009a74e-f8ba-425f-a9e4-0f2cc653ca9a tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Flavor pref 0:0:0 {{(pid=125641) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 19 00:42:52.599791 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.virt.hardware [None req-7009a74e-f8ba-425f-a9e4-0f2cc653ca9a tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Image pref 0:0:0 {{(pid=125641) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 19 00:42:52.600276 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.virt.hardware [None req-7009a74e-f8ba-425f-a9e4-0f2cc653ca9a tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=125641) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 19 00:42:52.611544 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-7009a74e-f8ba-425f-a9e4-0f2cc653ca9a tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:52.612031 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-7009a74e-f8ba-425f-a9e4-0f2cc653ca9a tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:52.612552 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-7009a74e-f8ba-425f-a9e4-0f2cc653ca9a tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:52.613297 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-7009a74e-f8ba-425f-a9e4-0f2cc653ca9a tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:52.613914 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-7009a74e-f8ba-425f-a9e4-0f2cc653ca9a tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:52.614353 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-7009a74e-f8ba-425f-a9e4-0f2cc653ca9a tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:52.633808 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.quota [None req-7009a74e-f8ba-425f-a9e4-0f2cc653ca9a tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Getting quotas for project c0151d58abd047a692f5cc4e7d05bea5. Resources: {'cores', 'ram', 'instances'} {{(pid=125641) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 19 00:42:52.639087 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.quota [None req-7009a74e-f8ba-425f-a9e4-0f2cc653ca9a tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Getting quotas for user d3f0d615187d4779bdd7da8c9f24c796 and project c0151d58abd047a692f5cc4e7d05bea5. Resources: {'cores', 'ram', 'instances'} {{(pid=125641) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 19 00:42:52.652561 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-7009a74e-f8ba-425f-a9e4-0f2cc653ca9a tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Going to run 1 instances... {{(pid=125641) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 19 00:42:52.656465 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-7009a74e-f8ba-425f-a9e4-0f2cc653ca9a tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] [instance: f6051269-1d16-42db-8323-67a532dbcf40] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='b387b4f8-f75e-4a68-b49c-a015bf61c46b',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=125641) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 19 00:42:52.711121 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-7009a74e-f8ba-425f-a9e4-0f2cc653ca9a tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 202 len: 409 microversion: 2.1 time: 1.021024 Oct 19 00:42:52.711944 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 150/265] 10.222.0.26 () {60 vars in 1147 bytes} [Wed Oct 19 00:42:51 2022] POST /compute/v2.1/servers => generated 409 bytes in 1023 msecs (HTTP/1.1 202) 10 headers in 451 bytes (2 switches on core 0) Oct 19 00:42:52.717699 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-55fd3607-d1cd-43d8-b017-5c743304a4e5 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:42:52.718727 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-55fd3607-d1cd-43d8-b017-5c743304a4e5 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] [instance: c5d02d64-d4e6-445d-a155-b875b8a8cce0] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:42:52.727174 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-55fd3607-d1cd-43d8-b017-5c743304a4e5 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:52.727987 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-55fd3607-d1cd-43d8-b017-5c743304a4e5 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:52.728434 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-55fd3607-d1cd-43d8-b017-5c743304a4e5 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:52.729218 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-55fd3607-d1cd-43d8-b017-5c743304a4e5 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:52.729867 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-55fd3607-d1cd-43d8-b017-5c743304a4e5 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:52.730301 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-55fd3607-d1cd-43d8-b017-5c743304a4e5 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:52.942384 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-55fd3607-d1cd-43d8-b017-5c743304a4e5 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a08c9457231a447eb1c4130e68cfb31d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '173a52b79dab4962a4aa380c26f97d2d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:42:52.955005 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-55fd3607-d1cd-43d8-b017-5c743304a4e5 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] 10.222.0.26 "GET /compute/v2.1/servers/c5d02d64-d4e6-445d-a155-b875b8a8cce0" status: 200 len: 1314 microversion: 2.1 time: 0.241467 Oct 19 00:42:52.955917 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 151/266] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:42:52 2022] GET /compute/v2.1/servers/c5d02d64-d4e6-445d-a155-b875b8a8cce0 => generated 1314 bytes in 243 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:42:52.961246 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-dfc370d4-25d7-453f-925a-68de8e90c152 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:42:52.962240 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-dfc370d4-25d7-453f-925a-68de8e90c152 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] [instance: b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:42:52.972029 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-dfc370d4-25d7-453f-925a-68de8e90c152 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:52.972805 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-dfc370d4-25d7-453f-925a-68de8e90c152 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:52.973680 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-dfc370d4-25d7-453f-925a-68de8e90c152 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:52.974534 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-dfc370d4-25d7-453f-925a-68de8e90c152 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:52.975325 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-dfc370d4-25d7-453f-925a-68de8e90c152 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:52.975891 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-dfc370d4-25d7-453f-925a-68de8e90c152 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:53.232716 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-dfc370d4-25d7-453f-925a-68de8e90c152 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:42:53.243080 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-dfc370d4-25d7-453f-925a-68de8e90c152 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:42:53.244911 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-dfc370d4-25d7-453f-925a-68de8e90c152 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:42:53.246122 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-dfc370d4-25d7-453f-925a-68de8e90c152 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] 10.222.0.26 "GET /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2" status: 200 len: 1723 microversion: 2.20 time: 0.289070 Oct 19 00:42:53.246720 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 152/267] 10.222.0.26 () {60 vars in 1277 bytes} [Wed Oct 19 00:42:52 2022] GET /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2 => generated 1723 bytes in 290 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 19 00:42:53.252746 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-475f4eb1-3d37-405c-952a-3c4b9e8fddd7 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:42:53.253355 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-475f4eb1-3d37-405c-952a-3c4b9e8fddd7 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] [instance: 25d64f9f-18a6-4b6a-92b1-3e8e7d017ab8] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:42:53.262964 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-475f4eb1-3d37-405c-952a-3c4b9e8fddd7 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:53.264565 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-475f4eb1-3d37-405c-952a-3c4b9e8fddd7 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:53.265185 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-475f4eb1-3d37-405c-952a-3c4b9e8fddd7 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:53.266775 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-475f4eb1-3d37-405c-952a-3c4b9e8fddd7 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:53.267758 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-475f4eb1-3d37-405c-952a-3c4b9e8fddd7 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:53.268513 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-475f4eb1-3d37-405c-952a-3c4b9e8fddd7 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:53.423860 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-475f4eb1-3d37-405c-952a-3c4b9e8fddd7 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f8725c4d05b4dd79b05b43fba15980b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0b35c2038f6b4d0a9012b004f1416c00', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:42:53.432858 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-475f4eb1-3d37-405c-952a-3c4b9e8fddd7 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] 10.222.0.26 "GET /compute/v2.1/servers/25d64f9f-18a6-4b6a-92b1-3e8e7d017ab8" status: 200 len: 1367 microversion: 2.1 time: 0.184828 Oct 19 00:42:53.433449 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 153/268] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:42:53 2022] GET /compute/v2.1/servers/25d64f9f-18a6-4b6a-92b1-3e8e7d017ab8 => generated 1367 bytes in 186 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:42:53.446749 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-10e6bd27-d99c-4963-a4e9-6804a76a2b9c tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:42:53.446749 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-10e6bd27-d99c-4963-a4e9-6804a76a2b9c tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] [instance: f6051269-1d16-42db-8323-67a532dbcf40] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:42:53.446749 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-10e6bd27-d99c-4963-a4e9-6804a76a2b9c tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:53.447309 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-10e6bd27-d99c-4963-a4e9-6804a76a2b9c tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:53.447827 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-10e6bd27-d99c-4963-a4e9-6804a76a2b9c tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:53.448735 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-10e6bd27-d99c-4963-a4e9-6804a76a2b9c tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:53.449695 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-10e6bd27-d99c-4963-a4e9-6804a76a2b9c tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:53.450139 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-10e6bd27-d99c-4963-a4e9-6804a76a2b9c tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:53.537814 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-10e6bd27-d99c-4963-a4e9-6804a76a2b9c tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd3f0d615187d4779bdd7da8c9f24c796', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0151d58abd047a692f5cc4e7d05bea5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:42:53.551533 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-10e6bd27-d99c-4963-a4e9-6804a76a2b9c tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] 10.222.0.26 "GET /compute/v2.1/servers/f6051269-1d16-42db-8323-67a532dbcf40" status: 200 len: 1250 microversion: 2.1 time: 0.116446 Oct 19 00:42:53.552437 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 154/269] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:42:53 2022] GET /compute/v2.1/servers/f6051269-1d16-42db-8323-67a532dbcf40 => generated 1250 bytes in 118 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:42:53.978922 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-ded3f21f-d49e-4eac-8d0f-4ac4e5908c73 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:42:53.980568 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-ded3f21f-d49e-4eac-8d0f-4ac4e5908c73 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] [instance: c5d02d64-d4e6-445d-a155-b875b8a8cce0] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:42:53.988200 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ded3f21f-d49e-4eac-8d0f-4ac4e5908c73 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:53.988694 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ded3f21f-d49e-4eac-8d0f-4ac4e5908c73 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:53.989127 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ded3f21f-d49e-4eac-8d0f-4ac4e5908c73 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:53.989830 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ded3f21f-d49e-4eac-8d0f-4ac4e5908c73 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:53.990285 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ded3f21f-d49e-4eac-8d0f-4ac4e5908c73 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:53.990849 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ded3f21f-d49e-4eac-8d0f-4ac4e5908c73 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:54.138317 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-ded3f21f-d49e-4eac-8d0f-4ac4e5908c73 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a08c9457231a447eb1c4130e68cfb31d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '173a52b79dab4962a4aa380c26f97d2d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:42:54.149057 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-ded3f21f-d49e-4eac-8d0f-4ac4e5908c73 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] 10.222.0.26 "GET /compute/v2.1/servers/c5d02d64-d4e6-445d-a155-b875b8a8cce0" status: 200 len: 1314 microversion: 2.1 time: 0.174969 Oct 19 00:42:54.150312 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 155/270] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:42:53 2022] GET /compute/v2.1/servers/c5d02d64-d4e6-445d-a155-b875b8a8cce0 => generated 1314 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 19 00:42:54.291142 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-80375596-62aa-4bc7-a7e2-2545be12cd16 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:42:54.291859 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-80375596-62aa-4bc7-a7e2-2545be12cd16 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] [instance: b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:42:54.300992 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-80375596-62aa-4bc7-a7e2-2545be12cd16 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:54.301582 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-80375596-62aa-4bc7-a7e2-2545be12cd16 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:54.302842 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-80375596-62aa-4bc7-a7e2-2545be12cd16 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:54.304142 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-80375596-62aa-4bc7-a7e2-2545be12cd16 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:54.305228 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-80375596-62aa-4bc7-a7e2-2545be12cd16 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:54.318872 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-80375596-62aa-4bc7-a7e2-2545be12cd16 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.014s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:54.735157 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-80375596-62aa-4bc7-a7e2-2545be12cd16 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:42:54.746342 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-80375596-62aa-4bc7-a7e2-2545be12cd16 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:42:54.748981 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-80375596-62aa-4bc7-a7e2-2545be12cd16 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:42:54.750463 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-80375596-62aa-4bc7-a7e2-2545be12cd16 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] 10.222.0.26 "GET /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2" status: 200 len: 1723 microversion: 2.20 time: 0.463469 Oct 19 00:42:54.761737 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 156/271] 10.222.0.26 () {60 vars in 1277 bytes} [Wed Oct 19 00:42:54 2022] GET /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2 => generated 1723 bytes in 475 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 19 00:42:54.783991 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-9a08a291-51ee-4830-b030-dc1145967e0f tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:42:54.784721 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-9a08a291-51ee-4830-b030-dc1145967e0f tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] [instance: 25d64f9f-18a6-4b6a-92b1-3e8e7d017ab8] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:42:54.794623 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-9a08a291-51ee-4830-b030-dc1145967e0f tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:54.795327 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-9a08a291-51ee-4830-b030-dc1145967e0f tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:54.796098 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-9a08a291-51ee-4830-b030-dc1145967e0f tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:54.796891 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-9a08a291-51ee-4830-b030-dc1145967e0f tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:54.797357 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-9a08a291-51ee-4830-b030-dc1145967e0f tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:54.798188 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-9a08a291-51ee-4830-b030-dc1145967e0f tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:55.069675 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-9a08a291-51ee-4830-b030-dc1145967e0f tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f8725c4d05b4dd79b05b43fba15980b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0b35c2038f6b4d0a9012b004f1416c00', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:42:55.080480 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-9a08a291-51ee-4830-b030-dc1145967e0f tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] 10.222.0.26 "GET /compute/v2.1/servers/25d64f9f-18a6-4b6a-92b1-3e8e7d017ab8" status: 200 len: 1367 microversion: 2.1 time: 0.308176 Oct 19 00:42:55.084580 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 157/272] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:42:54 2022] GET /compute/v2.1/servers/25d64f9f-18a6-4b6a-92b1-3e8e7d017ab8 => generated 1367 bytes in 321 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:42:55.092798 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-4f5c0268-60ac-4879-a221-ef55f4affdd4 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:42:55.093654 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-4f5c0268-60ac-4879-a221-ef55f4affdd4 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] [instance: f6051269-1d16-42db-8323-67a532dbcf40] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:42:55.101636 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-4f5c0268-60ac-4879-a221-ef55f4affdd4 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:55.102555 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-4f5c0268-60ac-4879-a221-ef55f4affdd4 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:55.103212 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-4f5c0268-60ac-4879-a221-ef55f4affdd4 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:55.104208 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-4f5c0268-60ac-4879-a221-ef55f4affdd4 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:55.104909 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-4f5c0268-60ac-4879-a221-ef55f4affdd4 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:55.105668 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-4f5c0268-60ac-4879-a221-ef55f4affdd4 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:55.112915 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG dbcounter [-] [125641] Writing DB stats nova_cell1:SELECT=100 {{(pid=125641) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 00:42:55.191593 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-4f5c0268-60ac-4879-a221-ef55f4affdd4 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd3f0d615187d4779bdd7da8c9f24c796', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0151d58abd047a692f5cc4e7d05bea5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:42:55.199987 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-4f5c0268-60ac-4879-a221-ef55f4affdd4 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] 10.222.0.26 "GET /compute/v2.1/servers/f6051269-1d16-42db-8323-67a532dbcf40" status: 200 len: 1298 microversion: 2.1 time: 0.114076 Oct 19 00:42:55.201466 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 158/273] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:42:55 2022] GET /compute/v2.1/servers/f6051269-1d16-42db-8323-67a532dbcf40 => generated 1298 bytes in 116 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:42:55.206457 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-d0771735-d162-4d54-98ba-558e12062552 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesSnapshotTestJSON-instance-1503598602", "imageRef": "b387b4f8-f75e-4a68-b49c-a015bf61c46b", "flavorRef": "452", "networks": [{"uuid": "febcf320-b179-4c31-a898-24a66a89b934"}]}} {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 19 00:42:55.276832 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.network.neutron [None req-d0771735-d162-4d54-98ba-558e12062552 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] validate_networks() for [('febcf320-b179-4c31-a898-24a66a89b934', None, None, None, None, None)] {{(pid=125641) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 19 00:42:55.768638 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.virt.hardware [None req-d0771735-d162-4d54-98ba-558e12062552 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Flavor limits 0:0:0 {{(pid=125641) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 19 00:42:55.769055 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.virt.hardware [None req-d0771735-d162-4d54-98ba-558e12062552 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Image limits 0:0:0 {{(pid=125641) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 19 00:42:55.769532 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.virt.hardware [None req-d0771735-d162-4d54-98ba-558e12062552 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Flavor pref 0:0:0 {{(pid=125641) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 19 00:42:55.770190 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.virt.hardware [None req-d0771735-d162-4d54-98ba-558e12062552 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Image pref 0:0:0 {{(pid=125641) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 19 00:42:55.770337 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.virt.hardware [None req-d0771735-d162-4d54-98ba-558e12062552 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=125641) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 19 00:42:55.780102 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d0771735-d162-4d54-98ba-558e12062552 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:55.780588 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d0771735-d162-4d54-98ba-558e12062552 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:55.781024 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d0771735-d162-4d54-98ba-558e12062552 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:55.781740 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d0771735-d162-4d54-98ba-558e12062552 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:55.782355 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d0771735-d162-4d54-98ba-558e12062552 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:55.782842 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d0771735-d162-4d54-98ba-558e12062552 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:55.803670 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.quota [None req-d0771735-d162-4d54-98ba-558e12062552 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Getting quotas for project 1235d8b054b3405595fca761f08b73ae. Resources: {'cores', 'ram', 'instances'} {{(pid=125641) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 19 00:42:55.807867 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.quota [None req-d0771735-d162-4d54-98ba-558e12062552 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Getting quotas for user 39de6a6fff7c4cc08cc62aa86182d71d and project 1235d8b054b3405595fca761f08b73ae. Resources: {'cores', 'ram', 'instances'} {{(pid=125641) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 19 00:42:55.821701 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-d0771735-d162-4d54-98ba-558e12062552 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Going to run 1 instances... {{(pid=125641) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 19 00:42:55.824366 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-d0771735-d162-4d54-98ba-558e12062552 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] [instance: 5bdc36c3-1db5-4ff2-808d-1295f314d522] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='b387b4f8-f75e-4a68-b49c-a015bf61c46b',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=125641) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 19 00:42:55.855178 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG dbcounter [-] [125641] Writing DB stats nova_api:INSERT=9,nova_api:SELECT=91 {{(pid=125641) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 00:42:55.879839 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-d0771735-d162-4d54-98ba-558e12062552 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.677259 Oct 19 00:42:55.882726 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 159/274] 10.222.0.26 () {60 vars in 1147 bytes} [Wed Oct 19 00:42:55 2022] POST /compute/v2.1/servers => generated 380 bytes in 681 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 19 00:42:55.895079 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-d5c3eef5-8fd6-4d3e-8343-b97c3fcac164 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:42:55.897088 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-d5c3eef5-8fd6-4d3e-8343-b97c3fcac164 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] [instance: c5d02d64-d4e6-445d-a155-b875b8a8cce0] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:42:55.913265 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d5c3eef5-8fd6-4d3e-8343-b97c3fcac164 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:55.913984 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d5c3eef5-8fd6-4d3e-8343-b97c3fcac164 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:55.914802 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d5c3eef5-8fd6-4d3e-8343-b97c3fcac164 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:55.916856 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d5c3eef5-8fd6-4d3e-8343-b97c3fcac164 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:55.918997 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d5c3eef5-8fd6-4d3e-8343-b97c3fcac164 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:55.920761 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d5c3eef5-8fd6-4d3e-8343-b97c3fcac164 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:56.158313 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-d5c3eef5-8fd6-4d3e-8343-b97c3fcac164 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a08c9457231a447eb1c4130e68cfb31d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '173a52b79dab4962a4aa380c26f97d2d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:42:56.169334 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-d5c3eef5-8fd6-4d3e-8343-b97c3fcac164 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] 10.222.0.26 "GET /compute/v2.1/servers/c5d02d64-d4e6-445d-a155-b875b8a8cce0" status: 200 len: 1497 microversion: 2.1 time: 0.284465 Oct 19 00:42:56.170391 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 160/275] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:42:55 2022] GET /compute/v2.1/servers/c5d02d64-d4e6-445d-a155-b875b8a8cce0 => generated 1497 bytes in 287 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:42:56.177553 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-c074e7c1-0417-4c60-9d3f-795854e016ba tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:42:56.178220 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-c074e7c1-0417-4c60-9d3f-795854e016ba tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] [instance: b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:42:56.184780 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-c074e7c1-0417-4c60-9d3f-795854e016ba tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:56.185800 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-c074e7c1-0417-4c60-9d3f-795854e016ba tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:56.187198 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-c074e7c1-0417-4c60-9d3f-795854e016ba tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:56.188235 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-c074e7c1-0417-4c60-9d3f-795854e016ba tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:56.188374 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-c074e7c1-0417-4c60-9d3f-795854e016ba tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:56.189230 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-c074e7c1-0417-4c60-9d3f-795854e016ba tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:56.415671 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-c074e7c1-0417-4c60-9d3f-795854e016ba tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:42:56.428961 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-c074e7c1-0417-4c60-9d3f-795854e016ba tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:42:56.437201 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-c074e7c1-0417-4c60-9d3f-795854e016ba tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:42:56.442111 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-c074e7c1-0417-4c60-9d3f-795854e016ba tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] 10.222.0.26 "GET /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2" status: 200 len: 1744 microversion: 2.20 time: 0.268578 Oct 19 00:42:56.444574 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 161/276] 10.222.0.26 () {60 vars in 1277 bytes} [Wed Oct 19 00:42:56 2022] GET /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2 => generated 1744 bytes in 273 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 19 00:42:56.463463 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-1ddb5348-6957-4dc4-9fbd-fd37d2efc09b tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:42:56.469444 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-1ddb5348-6957-4dc4-9fbd-fd37d2efc09b tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] [instance: 5bdc36c3-1db5-4ff2-808d-1295f314d522] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:42:56.608243 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-1ddb5348-6957-4dc4-9fbd-fd37d2efc09b tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '39de6a6fff7c4cc08cc62aa86182d71d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1235d8b054b3405595fca761f08b73ae', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:42:56.617669 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-1ddb5348-6957-4dc4-9fbd-fd37d2efc09b tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] 10.222.0.26 "GET /compute/v2.1/servers/5bdc36c3-1db5-4ff2-808d-1295f314d522" status: 200 len: 1219 microversion: 2.1 time: 0.170615 Oct 19 00:42:56.620642 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 162/277] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:42:56 2022] GET /compute/v2.1/servers/5bdc36c3-1db5-4ff2-808d-1295f314d522 => generated 1219 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:42:56.635008 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-b039de95-e7cc-49f1-bf7d-830ade5f58c5 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:42:56.635769 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-b039de95-e7cc-49f1-bf7d-830ade5f58c5 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] [instance: 25d64f9f-18a6-4b6a-92b1-3e8e7d017ab8] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:42:56.645619 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-b039de95-e7cc-49f1-bf7d-830ade5f58c5 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:56.646113 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-b039de95-e7cc-49f1-bf7d-830ade5f58c5 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:56.653112 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-b039de95-e7cc-49f1-bf7d-830ade5f58c5 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:56.655202 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-b039de95-e7cc-49f1-bf7d-830ade5f58c5 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:56.655762 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-b039de95-e7cc-49f1-bf7d-830ade5f58c5 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:56.656276 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-b039de95-e7cc-49f1-bf7d-830ade5f58c5 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:57.118604 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-b039de95-e7cc-49f1-bf7d-830ade5f58c5 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f8725c4d05b4dd79b05b43fba15980b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0b35c2038f6b4d0a9012b004f1416c00', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:42:57.132231 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-b039de95-e7cc-49f1-bf7d-830ade5f58c5 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] 10.222.0.26 "GET /compute/v2.1/servers/25d64f9f-18a6-4b6a-92b1-3e8e7d017ab8" status: 200 len: 1367 microversion: 2.1 time: 0.510382 Oct 19 00:42:57.132840 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 163/278] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:42:56 2022] GET /compute/v2.1/servers/25d64f9f-18a6-4b6a-92b1-3e8e7d017ab8 => generated 1367 bytes in 512 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:42:57.164006 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-c1ebe33a-a1d8-4e69-8055-86cba0e12e1f tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:42:57.164632 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-c1ebe33a-a1d8-4e69-8055-86cba0e12e1f tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] [instance: f6051269-1d16-42db-8323-67a532dbcf40] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:42:57.191895 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-c1ebe33a-a1d8-4e69-8055-86cba0e12e1f tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:57.193637 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-c1ebe33a-a1d8-4e69-8055-86cba0e12e1f tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:57.194321 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-c1ebe33a-a1d8-4e69-8055-86cba0e12e1f tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:57.194873 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-c1ebe33a-a1d8-4e69-8055-86cba0e12e1f tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:57.195344 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-c1ebe33a-a1d8-4e69-8055-86cba0e12e1f tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:57.195772 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-c1ebe33a-a1d8-4e69-8055-86cba0e12e1f tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:57.350552 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-c1ebe33a-a1d8-4e69-8055-86cba0e12e1f tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd3f0d615187d4779bdd7da8c9f24c796', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0151d58abd047a692f5cc4e7d05bea5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:42:57.364074 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-c1ebe33a-a1d8-4e69-8055-86cba0e12e1f tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] 10.222.0.26 "GET /compute/v2.1/servers/f6051269-1d16-42db-8323-67a532dbcf40" status: 200 len: 1304 microversion: 2.1 time: 0.227215 Oct 19 00:42:57.378913 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 164/279] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:42:57 2022] GET /compute/v2.1/servers/f6051269-1d16-42db-8323-67a532dbcf40 => generated 1304 bytes in 245 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:42:57.386122 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-7caa6aba-53a8-4095-a4d0-97b8e9b20cd8 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:42:57.387061 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-7caa6aba-53a8-4095-a4d0-97b8e9b20cd8 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] [instance: c5d02d64-d4e6-445d-a155-b875b8a8cce0] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:42:57.397584 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-7caa6aba-53a8-4095-a4d0-97b8e9b20cd8 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:57.398273 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-7caa6aba-53a8-4095-a4d0-97b8e9b20cd8 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:57.399044 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-7caa6aba-53a8-4095-a4d0-97b8e9b20cd8 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:57.400532 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-7caa6aba-53a8-4095-a4d0-97b8e9b20cd8 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:57.401001 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-7caa6aba-53a8-4095-a4d0-97b8e9b20cd8 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:57.401475 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-7caa6aba-53a8-4095-a4d0-97b8e9b20cd8 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:57.465796 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-7caa6aba-53a8-4095-a4d0-97b8e9b20cd8 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] [instance: c5d02d64-d4e6-445d-a155-b875b8a8cce0] Going to try to terminate instance {{(pid=125641) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 19 00:42:57.625987 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-7caa6aba-53a8-4095-a4d0-97b8e9b20cd8 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] 10.222.0.26 "DELETE /compute/v2.1/servers/c5d02d64-d4e6-445d-a155-b875b8a8cce0" status: 204 len: 0 microversion: 2.1 time: 0.244822 Oct 19 00:42:57.628168 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 165/280] 10.222.0.26 () {58 vars in 1239 bytes} [Wed Oct 19 00:42:57 2022] DELETE /compute/v2.1/servers/c5d02d64-d4e6-445d-a155-b875b8a8cce0 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 19 00:42:57.644727 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [req-9a86c6c4-b13a-4622-84c4-2c876acc495b req-b6a7a8fe-efaa-4134-ab0f-2f44ac371056 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2", "name": "network-vif-plugged", "status": "completed", "tag": "6aea38b4-b7a0-4e8e-82dc-deb19c260092"}]} {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 19 00:42:57.667075 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-9a86c6c4-b13a-4622-84c4-2c876acc495b req-b6a7a8fe-efaa-4134-ab0f-2f44ac371056 service nova] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:57.671343 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-9a86c6c4-b13a-4622-84c4-2c876acc495b req-b6a7a8fe-efaa-4134-ab0f-2f44ac371056 service nova] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:57.672080 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-9a86c6c4-b13a-4622-84c4-2c876acc495b req-b6a7a8fe-efaa-4134-ab0f-2f44ac371056 service nova] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:57.696278 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.compute.server_external_events [req-9a86c6c4-b13a-4622-84c4-2c876acc495b req-b6a7a8fe-efaa-4134-ab0f-2f44ac371056 service nova] Creating event network-vif-plugged:6aea38b4-b7a0-4e8e-82dc-deb19c260092 for instance b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2 on hv-cinder-maste Oct 19 00:42:57.704162 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [req-9a86c6c4-b13a-4622-84c4-2c876acc495b req-b6a7a8fe-efaa-4134-ab0f-2f44ac371056 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.068355 Oct 19 00:42:57.704987 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 166/281] 10.222.0.26 () {64 vars in 1317 bytes} [Wed Oct 19 00:42:57 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 71 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 19 00:42:57.712861 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-3974eb18-9f01-4cd4-adbf-5adacbdeebb1 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:42:57.713950 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-3974eb18-9f01-4cd4-adbf-5adacbdeebb1 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] [instance: b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:42:57.722726 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-3974eb18-9f01-4cd4-adbf-5adacbdeebb1 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:57.723570 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-3974eb18-9f01-4cd4-adbf-5adacbdeebb1 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:57.724185 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-3974eb18-9f01-4cd4-adbf-5adacbdeebb1 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:57.725058 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-3974eb18-9f01-4cd4-adbf-5adacbdeebb1 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:57.725731 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-3974eb18-9f01-4cd4-adbf-5adacbdeebb1 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:57.726312 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-3974eb18-9f01-4cd4-adbf-5adacbdeebb1 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:57.937688 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-3974eb18-9f01-4cd4-adbf-5adacbdeebb1 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:42:57.946517 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-3974eb18-9f01-4cd4-adbf-5adacbdeebb1 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:42:57.948299 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-3974eb18-9f01-4cd4-adbf-5adacbdeebb1 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:42:57.949200 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-3974eb18-9f01-4cd4-adbf-5adacbdeebb1 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] 10.222.0.26 "GET /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2" status: 200 len: 1744 microversion: 2.20 time: 0.242760 Oct 19 00:42:57.949939 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 167/282] 10.222.0.26 () {60 vars in 1277 bytes} [Wed Oct 19 00:42:57 2022] GET /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2 => generated 1744 bytes in 244 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 19 00:42:57.955423 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-53657cc8-22fa-4ef2-8eae-51e186ac3d7e tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:42:57.956028 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-53657cc8-22fa-4ef2-8eae-51e186ac3d7e tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] [instance: c5d02d64-d4e6-445d-a155-b875b8a8cce0] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:42:57.961954 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-53657cc8-22fa-4ef2-8eae-51e186ac3d7e tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:57.962437 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-53657cc8-22fa-4ef2-8eae-51e186ac3d7e tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:57.963185 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-53657cc8-22fa-4ef2-8eae-51e186ac3d7e tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:57.964044 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-53657cc8-22fa-4ef2-8eae-51e186ac3d7e tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:57.964877 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-53657cc8-22fa-4ef2-8eae-51e186ac3d7e tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:57.965454 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-53657cc8-22fa-4ef2-8eae-51e186ac3d7e tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:58.288357 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-53657cc8-22fa-4ef2-8eae-51e186ac3d7e tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a08c9457231a447eb1c4130e68cfb31d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '173a52b79dab4962a4aa380c26f97d2d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:42:58.299989 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-53657cc8-22fa-4ef2-8eae-51e186ac3d7e tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] 10.222.0.26 "GET /compute/v2.1/servers/c5d02d64-d4e6-445d-a155-b875b8a8cce0" status: 200 len: 1503 microversion: 2.1 time: 0.349039 Oct 19 00:42:58.300622 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 168/283] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:42:57 2022] GET /compute/v2.1/servers/c5d02d64-d4e6-445d-a155-b875b8a8cce0 => generated 1503 bytes in 350 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:42:58.320272 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-fab065e3-1ff5-4fba-b403-d0445dd8b907 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:42:58.321493 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-fab065e3-1ff5-4fba-b403-d0445dd8b907 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] [instance: 5bdc36c3-1db5-4ff2-808d-1295f314d522] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:42:58.333695 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-fab065e3-1ff5-4fba-b403-d0445dd8b907 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:58.333695 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-fab065e3-1ff5-4fba-b403-d0445dd8b907 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:58.334779 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-fab065e3-1ff5-4fba-b403-d0445dd8b907 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:58.336356 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-fab065e3-1ff5-4fba-b403-d0445dd8b907 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:58.336908 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-fab065e3-1ff5-4fba-b403-d0445dd8b907 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:58.337527 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-fab065e3-1ff5-4fba-b403-d0445dd8b907 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:58.458576 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-fab065e3-1ff5-4fba-b403-d0445dd8b907 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '39de6a6fff7c4cc08cc62aa86182d71d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1235d8b054b3405595fca761f08b73ae', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:42:58.471968 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-fab065e3-1ff5-4fba-b403-d0445dd8b907 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] 10.222.0.26 "GET /compute/v2.1/servers/5bdc36c3-1db5-4ff2-808d-1295f314d522" status: 200 len: 1271 microversion: 2.1 time: 0.158405 Oct 19 00:42:58.487649 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 169/284] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:42:58 2022] GET /compute/v2.1/servers/5bdc36c3-1db5-4ff2-808d-1295f314d522 => generated 1271 bytes in 186 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:42:58.496216 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-20fc1f2e-1bf4-43b0-8ab0-664ca35fd213 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:42:58.497362 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-20fc1f2e-1bf4-43b0-8ab0-664ca35fd213 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] [instance: 25d64f9f-18a6-4b6a-92b1-3e8e7d017ab8] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:42:58.509324 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-20fc1f2e-1bf4-43b0-8ab0-664ca35fd213 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:58.509792 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-20fc1f2e-1bf4-43b0-8ab0-664ca35fd213 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:58.510933 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-20fc1f2e-1bf4-43b0-8ab0-664ca35fd213 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:58.511947 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-20fc1f2e-1bf4-43b0-8ab0-664ca35fd213 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:58.512510 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-20fc1f2e-1bf4-43b0-8ab0-664ca35fd213 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:58.513003 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-20fc1f2e-1bf4-43b0-8ab0-664ca35fd213 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:58.678552 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-20fc1f2e-1bf4-43b0-8ab0-664ca35fd213 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f8725c4d05b4dd79b05b43fba15980b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0b35c2038f6b4d0a9012b004f1416c00', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:42:58.686692 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-20fc1f2e-1bf4-43b0-8ab0-664ca35fd213 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] 10.222.0.26 "GET /compute/v2.1/servers/25d64f9f-18a6-4b6a-92b1-3e8e7d017ab8" status: 200 len: 1367 microversion: 2.1 time: 0.197627 Oct 19 00:42:58.687312 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 170/285] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:42:58 2022] GET /compute/v2.1/servers/25d64f9f-18a6-4b6a-92b1-3e8e7d017ab8 => generated 1367 bytes in 199 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:42:58.696200 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-bba10c5a-a57f-485a-9007-c0846731a0c8 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:42:58.697488 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-bba10c5a-a57f-485a-9007-c0846731a0c8 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] [instance: f6051269-1d16-42db-8323-67a532dbcf40] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:42:58.705909 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-bba10c5a-a57f-485a-9007-c0846731a0c8 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:58.706968 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-bba10c5a-a57f-485a-9007-c0846731a0c8 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:58.707487 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-bba10c5a-a57f-485a-9007-c0846731a0c8 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:58.708354 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-bba10c5a-a57f-485a-9007-c0846731a0c8 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:58.708882 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-bba10c5a-a57f-485a-9007-c0846731a0c8 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:58.709374 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-bba10c5a-a57f-485a-9007-c0846731a0c8 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:58.915221 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-bba10c5a-a57f-485a-9007-c0846731a0c8 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd3f0d615187d4779bdd7da8c9f24c796', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0151d58abd047a692f5cc4e7d05bea5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:42:58.924570 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-bba10c5a-a57f-485a-9007-c0846731a0c8 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] 10.222.0.26 "GET /compute/v2.1/servers/f6051269-1d16-42db-8323-67a532dbcf40" status: 200 len: 1372 microversion: 2.1 time: 0.232712 Oct 19 00:42:58.925193 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 171/286] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:42:58 2022] GET /compute/v2.1/servers/f6051269-1d16-42db-8323-67a532dbcf40 => generated 1372 bytes in 237 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:42:58.980387 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-fa92e5fc-7cf2-45ce-bf12-1b37976c8d85 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:42:58.981018 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-fa92e5fc-7cf2-45ce-bf12-1b37976c8d85 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] [instance: b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:42:58.987324 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-fa92e5fc-7cf2-45ce-bf12-1b37976c8d85 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:58.987824 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-fa92e5fc-7cf2-45ce-bf12-1b37976c8d85 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:58.988269 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-fa92e5fc-7cf2-45ce-bf12-1b37976c8d85 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:58.988970 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-fa92e5fc-7cf2-45ce-bf12-1b37976c8d85 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:58.989437 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-fa92e5fc-7cf2-45ce-bf12-1b37976c8d85 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:58.989866 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-fa92e5fc-7cf2-45ce-bf12-1b37976c8d85 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:59.145782 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-fa92e5fc-7cf2-45ce-bf12-1b37976c8d85 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:42:59.155329 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-fa92e5fc-7cf2-45ce-bf12-1b37976c8d85 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:42:59.156819 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-fa92e5fc-7cf2-45ce-bf12-1b37976c8d85 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:42:59.158008 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-fa92e5fc-7cf2-45ce-bf12-1b37976c8d85 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] 10.222.0.26 "GET /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2" status: 200 len: 1744 microversion: 2.20 time: 0.181754 Oct 19 00:42:59.158654 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 172/287] 10.222.0.26 () {60 vars in 1277 bytes} [Wed Oct 19 00:42:58 2022] GET /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2 => generated 1744 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 19 00:42:59.328182 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-641408f8-f208-473e-b4d6-949541fa2530 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:42:59.328966 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-641408f8-f208-473e-b4d6-949541fa2530 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] [instance: c5d02d64-d4e6-445d-a155-b875b8a8cce0] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:42:59.335158 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-641408f8-f208-473e-b4d6-949541fa2530 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:59.335636 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-641408f8-f208-473e-b4d6-949541fa2530 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:59.336145 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-641408f8-f208-473e-b4d6-949541fa2530 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:59.336906 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-641408f8-f208-473e-b4d6-949541fa2530 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:59.337425 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-641408f8-f208-473e-b4d6-949541fa2530 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:59.337917 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-641408f8-f208-473e-b4d6-949541fa2530 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:59.489086 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-641408f8-f208-473e-b4d6-949541fa2530 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a08c9457231a447eb1c4130e68cfb31d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '173a52b79dab4962a4aa380c26f97d2d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:42:59.497927 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-641408f8-f208-473e-b4d6-949541fa2530 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] 10.222.0.26 "GET /compute/v2.1/servers/c5d02d64-d4e6-445d-a155-b875b8a8cce0" status: 200 len: 1503 microversion: 2.1 time: 0.174126 Oct 19 00:42:59.498885 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 173/288] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:42:59 2022] GET /compute/v2.1/servers/c5d02d64-d4e6-445d-a155-b875b8a8cce0 => generated 1503 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 19 00:42:59.503789 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-533bff29-e6e7-487e-8670-674a3cc3b7f0 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:42:59.504423 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-533bff29-e6e7-487e-8670-674a3cc3b7f0 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] [instance: 5bdc36c3-1db5-4ff2-808d-1295f314d522] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:42:59.510617 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-533bff29-e6e7-487e-8670-674a3cc3b7f0 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:59.511080 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-533bff29-e6e7-487e-8670-674a3cc3b7f0 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:59.511504 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-533bff29-e6e7-487e-8670-674a3cc3b7f0 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:59.512433 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-533bff29-e6e7-487e-8670-674a3cc3b7f0 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:59.512856 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-533bff29-e6e7-487e-8670-674a3cc3b7f0 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:59.513942 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-533bff29-e6e7-487e-8670-674a3cc3b7f0 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:59.583456 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-533bff29-e6e7-487e-8670-674a3cc3b7f0 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '39de6a6fff7c4cc08cc62aa86182d71d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1235d8b054b3405595fca761f08b73ae', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:42:59.591998 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-533bff29-e6e7-487e-8670-674a3cc3b7f0 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] 10.222.0.26 "GET /compute/v2.1/servers/5bdc36c3-1db5-4ff2-808d-1295f314d522" status: 200 len: 1271 microversion: 2.1 time: 0.091417 Oct 19 00:42:59.592529 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 174/289] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:42:59 2022] GET /compute/v2.1/servers/5bdc36c3-1db5-4ff2-808d-1295f314d522 => generated 1271 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:42:59.709297 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-a3597b0e-13ac-4a96-9fa8-0869808b28b1 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:42:59.709908 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-a3597b0e-13ac-4a96-9fa8-0869808b28b1 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] [instance: 25d64f9f-18a6-4b6a-92b1-3e8e7d017ab8] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:42:59.715921 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-a3597b0e-13ac-4a96-9fa8-0869808b28b1 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:59.716486 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-a3597b0e-13ac-4a96-9fa8-0869808b28b1 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:59.717385 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-a3597b0e-13ac-4a96-9fa8-0869808b28b1 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:59.718096 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-a3597b0e-13ac-4a96-9fa8-0869808b28b1 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:59.718583 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-a3597b0e-13ac-4a96-9fa8-0869808b28b1 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:59.719003 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-a3597b0e-13ac-4a96-9fa8-0869808b28b1 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:59.740671 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG dbcounter [-] [125641] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=125641) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 00:42:59.865210 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-a3597b0e-13ac-4a96-9fa8-0869808b28b1 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f8725c4d05b4dd79b05b43fba15980b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0b35c2038f6b4d0a9012b004f1416c00', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:42:59.877340 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-a3597b0e-13ac-4a96-9fa8-0869808b28b1 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] 10.222.0.26 "GET /compute/v2.1/servers/25d64f9f-18a6-4b6a-92b1-3e8e7d017ab8" status: 200 len: 1367 microversion: 2.1 time: 0.171890 Oct 19 00:42:59.877910 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 175/290] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:42:59 2022] GET /compute/v2.1/servers/25d64f9f-18a6-4b6a-92b1-3e8e7d017ab8 => generated 1367 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 19 00:42:59.947489 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-d58789ec-d040-471c-b5e0-fd672d20cb8e tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:42:59.948233 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-d58789ec-d040-471c-b5e0-fd672d20cb8e tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] [instance: f6051269-1d16-42db-8323-67a532dbcf40] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:42:59.955857 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d58789ec-d040-471c-b5e0-fd672d20cb8e tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:59.956512 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d58789ec-d040-471c-b5e0-fd672d20cb8e tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:59.957098 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d58789ec-d040-471c-b5e0-fd672d20cb8e tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:42:59.957846 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d58789ec-d040-471c-b5e0-fd672d20cb8e tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:42:59.958317 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d58789ec-d040-471c-b5e0-fd672d20cb8e tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:42:59.958789 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d58789ec-d040-471c-b5e0-fd672d20cb8e tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:43:00.106580 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-d58789ec-d040-471c-b5e0-fd672d20cb8e tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd3f0d615187d4779bdd7da8c9f24c796', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0151d58abd047a692f5cc4e7d05bea5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:43:00.115552 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-d58789ec-d040-471c-b5e0-fd672d20cb8e tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] 10.222.0.26 "GET /compute/v2.1/servers/f6051269-1d16-42db-8323-67a532dbcf40" status: 200 len: 1372 microversion: 2.1 time: 0.171985 Oct 19 00:43:00.116058 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 176/291] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:42:59 2022] GET /compute/v2.1/servers/f6051269-1d16-42db-8323-67a532dbcf40 => generated 1372 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 19 00:43:00.184668 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-953da122-f143-46ce-9195-7c40e4f9badc tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:43:00.185922 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-953da122-f143-46ce-9195-7c40e4f9badc tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] [instance: b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:43:00.192807 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-953da122-f143-46ce-9195-7c40e4f9badc tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:43:00.193305 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-953da122-f143-46ce-9195-7c40e4f9badc tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:43:00.193714 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-953da122-f143-46ce-9195-7c40e4f9badc tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:43:00.199038 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-953da122-f143-46ce-9195-7c40e4f9badc tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:43:00.199038 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-953da122-f143-46ce-9195-7c40e4f9badc tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:43:00.199038 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-953da122-f143-46ce-9195-7c40e4f9badc tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:43:00.340847 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-953da122-f143-46ce-9195-7c40e4f9badc tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:43:00.350054 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-953da122-f143-46ce-9195-7c40e4f9badc tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:43:00.351546 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-953da122-f143-46ce-9195-7c40e4f9badc tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:43:00.352743 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-953da122-f143-46ce-9195-7c40e4f9badc tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] 10.222.0.26 "GET /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2" status: 200 len: 1744 microversion: 2.20 time: 0.172207 Oct 19 00:43:00.353298 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 177/292] 10.222.0.26 () {60 vars in 1277 bytes} [Wed Oct 19 00:43:00 2022] GET /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2 => generated 1744 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 19 00:43:00.523995 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-ee566735-759d-4f85-b9e2-c0ab5bd6987b tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:43:00.524609 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-ee566735-759d-4f85-b9e2-c0ab5bd6987b tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] [instance: c5d02d64-d4e6-445d-a155-b875b8a8cce0] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:43:00.531360 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ee566735-759d-4f85-b9e2-c0ab5bd6987b tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:43:00.531839 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ee566735-759d-4f85-b9e2-c0ab5bd6987b tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:43:00.532270 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ee566735-759d-4f85-b9e2-c0ab5bd6987b tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:43:00.532960 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ee566735-759d-4f85-b9e2-c0ab5bd6987b tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:43:00.533415 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ee566735-759d-4f85-b9e2-c0ab5bd6987b tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:43:00.533834 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ee566735-759d-4f85-b9e2-c0ab5bd6987b tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:43:00.669299 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-ee566735-759d-4f85-b9e2-c0ab5bd6987b tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a08c9457231a447eb1c4130e68cfb31d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '173a52b79dab4962a4aa380c26f97d2d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:43:00.677276 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-ee566735-759d-4f85-b9e2-c0ab5bd6987b tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] 10.222.0.26 "GET /compute/v2.1/servers/c5d02d64-d4e6-445d-a155-b875b8a8cce0" status: 200 len: 1503 microversion: 2.1 time: 0.157079 Oct 19 00:43:00.677856 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 178/293] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:43:00 2022] GET /compute/v2.1/servers/c5d02d64-d4e6-445d-a155-b875b8a8cce0 => generated 1503 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 19 00:43:00.682334 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-82a21b5d-98c0-4413-a632-4a16aaf8e045 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:43:00.682973 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-82a21b5d-98c0-4413-a632-4a16aaf8e045 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] [instance: 5bdc36c3-1db5-4ff2-808d-1295f314d522] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:43:00.689548 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-82a21b5d-98c0-4413-a632-4a16aaf8e045 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:43:00.690039 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-82a21b5d-98c0-4413-a632-4a16aaf8e045 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:43:00.690494 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-82a21b5d-98c0-4413-a632-4a16aaf8e045 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:43:00.691214 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-82a21b5d-98c0-4413-a632-4a16aaf8e045 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:43:00.691670 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-82a21b5d-98c0-4413-a632-4a16aaf8e045 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:43:00.692097 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-82a21b5d-98c0-4413-a632-4a16aaf8e045 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:43:00.757175 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-82a21b5d-98c0-4413-a632-4a16aaf8e045 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '39de6a6fff7c4cc08cc62aa86182d71d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1235d8b054b3405595fca761f08b73ae', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:43:00.766667 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-82a21b5d-98c0-4413-a632-4a16aaf8e045 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] 10.222.0.26 "GET /compute/v2.1/servers/5bdc36c3-1db5-4ff2-808d-1295f314d522" status: 200 len: 1271 microversion: 2.1 time: 0.087598 Oct 19 00:43:00.767172 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 179/294] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:43:00 2022] GET /compute/v2.1/servers/5bdc36c3-1db5-4ff2-808d-1295f314d522 => generated 1271 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:43:00.901021 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-8e0ea06a-949c-4945-9187-84cb6e946751 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:43:00.901541 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-8e0ea06a-949c-4945-9187-84cb6e946751 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] [instance: 25d64f9f-18a6-4b6a-92b1-3e8e7d017ab8] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:43:00.908322 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-8e0ea06a-949c-4945-9187-84cb6e946751 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:43:00.908782 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-8e0ea06a-949c-4945-9187-84cb6e946751 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:43:00.909208 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-8e0ea06a-949c-4945-9187-84cb6e946751 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:43:00.909896 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-8e0ea06a-949c-4945-9187-84cb6e946751 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:43:00.910343 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-8e0ea06a-949c-4945-9187-84cb6e946751 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:43:00.910794 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-8e0ea06a-949c-4945-9187-84cb6e946751 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:43:01.074449 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-8e0ea06a-949c-4945-9187-84cb6e946751 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f8725c4d05b4dd79b05b43fba15980b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0b35c2038f6b4d0a9012b004f1416c00', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:43:01.083772 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-8e0ea06a-949c-4945-9187-84cb6e946751 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] 10.222.0.26 "GET /compute/v2.1/servers/25d64f9f-18a6-4b6a-92b1-3e8e7d017ab8" status: 200 len: 1367 microversion: 2.1 time: 0.186815 Oct 19 00:43:01.084359 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 180/295] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:43:00 2022] GET /compute/v2.1/servers/25d64f9f-18a6-4b6a-92b1-3e8e7d017ab8 => generated 1367 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 19 00:43:01.137781 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-20763441-e5bd-4274-9aad-df7f8f84962a tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:43:01.139106 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-20763441-e5bd-4274-9aad-df7f8f84962a tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] [instance: f6051269-1d16-42db-8323-67a532dbcf40] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:43:01.145548 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-20763441-e5bd-4274-9aad-df7f8f84962a tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:43:01.146122 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-20763441-e5bd-4274-9aad-df7f8f84962a tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:43:01.146660 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-20763441-e5bd-4274-9aad-df7f8f84962a tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:43:01.147403 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-20763441-e5bd-4274-9aad-df7f8f84962a tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:43:01.147863 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-20763441-e5bd-4274-9aad-df7f8f84962a tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:43:01.148296 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-20763441-e5bd-4274-9aad-df7f8f84962a tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:43:01.288470 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-20763441-e5bd-4274-9aad-df7f8f84962a tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd3f0d615187d4779bdd7da8c9f24c796', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0151d58abd047a692f5cc4e7d05bea5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:43:01.297133 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-20763441-e5bd-4274-9aad-df7f8f84962a tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] 10.222.0.26 "GET /compute/v2.1/servers/f6051269-1d16-42db-8323-67a532dbcf40" status: 200 len: 1372 microversion: 2.1 time: 0.163330 Oct 19 00:43:01.297710 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 181/296] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:43:01 2022] GET /compute/v2.1/servers/f6051269-1d16-42db-8323-67a532dbcf40 => generated 1372 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 19 00:43:01.378833 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-4bb2f396-d3a7-4160-ad6b-66f312b5f7d1 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:43:01.379609 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-4bb2f396-d3a7-4160-ad6b-66f312b5f7d1 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] [instance: b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:43:01.385496 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-4bb2f396-d3a7-4160-ad6b-66f312b5f7d1 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:43:01.385977 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-4bb2f396-d3a7-4160-ad6b-66f312b5f7d1 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:43:01.386548 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-4bb2f396-d3a7-4160-ad6b-66f312b5f7d1 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:43:01.387255 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-4bb2f396-d3a7-4160-ad6b-66f312b5f7d1 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:43:01.388124 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-4bb2f396-d3a7-4160-ad6b-66f312b5f7d1 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:43:01.388124 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-4bb2f396-d3a7-4160-ad6b-66f312b5f7d1 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:43:01.604805 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-4bb2f396-d3a7-4160-ad6b-66f312b5f7d1 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:43:01.615583 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-4bb2f396-d3a7-4160-ad6b-66f312b5f7d1 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:43:01.617184 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-4bb2f396-d3a7-4160-ad6b-66f312b5f7d1 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:43:01.618422 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-4bb2f396-d3a7-4160-ad6b-66f312b5f7d1 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] 10.222.0.26 "GET /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2" status: 200 len: 1744 microversion: 2.20 time: 0.243656 Oct 19 00:43:01.619040 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 182/297] 10.222.0.26 () {60 vars in 1277 bytes} [Wed Oct 19 00:43:01 2022] GET /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2 => generated 1744 bytes in 245 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 19 00:43:01.702970 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-01545f40-6209-4f2f-a099-91783025075f tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:43:01.703959 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-01545f40-6209-4f2f-a099-91783025075f tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] [instance: c5d02d64-d4e6-445d-a155-b875b8a8cce0] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:43:01.715498 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-01545f40-6209-4f2f-a099-91783025075f tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:43:01.717145 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-01545f40-6209-4f2f-a099-91783025075f tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:43:01.717688 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-01545f40-6209-4f2f-a099-91783025075f tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:43:01.718843 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-01545f40-6209-4f2f-a099-91783025075f tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:43:01.719675 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-01545f40-6209-4f2f-a099-91783025075f tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:43:01.720605 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-01545f40-6209-4f2f-a099-91783025075f tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:43:01.970553 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-01545f40-6209-4f2f-a099-91783025075f tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a08c9457231a447eb1c4130e68cfb31d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '173a52b79dab4962a4aa380c26f97d2d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:43:01.984144 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-01545f40-6209-4f2f-a099-91783025075f tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] 10.222.0.26 "GET /compute/v2.1/servers/c5d02d64-d4e6-445d-a155-b875b8a8cce0" status: 200 len: 1503 microversion: 2.1 time: 0.285968 Oct 19 00:43:01.999865 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 183/298] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:43:01 2022] GET /compute/v2.1/servers/c5d02d64-d4e6-445d-a155-b875b8a8cce0 => generated 1503 bytes in 303 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 19 00:43:02.021148 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-a7ca1fb4-dcc6-41e1-a77a-7255eb5ebfe1 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:43:02.021761 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-a7ca1fb4-dcc6-41e1-a77a-7255eb5ebfe1 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] [instance: 5bdc36c3-1db5-4ff2-808d-1295f314d522] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:43:02.040183 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-a7ca1fb4-dcc6-41e1-a77a-7255eb5ebfe1 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:43:02.040679 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-a7ca1fb4-dcc6-41e1-a77a-7255eb5ebfe1 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:43:02.041136 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-a7ca1fb4-dcc6-41e1-a77a-7255eb5ebfe1 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:43:02.041810 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-a7ca1fb4-dcc6-41e1-a77a-7255eb5ebfe1 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:43:02.042468 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-a7ca1fb4-dcc6-41e1-a77a-7255eb5ebfe1 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:43:02.049357 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-a7ca1fb4-dcc6-41e1-a77a-7255eb5ebfe1 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:43:02.178288 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-a7ca1fb4-dcc6-41e1-a77a-7255eb5ebfe1 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '39de6a6fff7c4cc08cc62aa86182d71d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1235d8b054b3405595fca761f08b73ae', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:43:02.187349 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-a7ca1fb4-dcc6-41e1-a77a-7255eb5ebfe1 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] 10.222.0.26 "GET /compute/v2.1/servers/5bdc36c3-1db5-4ff2-808d-1295f314d522" status: 200 len: 1271 microversion: 2.1 time: 0.186228 Oct 19 00:43:02.187966 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 184/299] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:43:01 2022] GET /compute/v2.1/servers/5bdc36c3-1db5-4ff2-808d-1295f314d522 => generated 1271 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:43:02.205093 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [req-5b00f052-f764-48f0-82e5-2a98356e7458 req-36ad9df6-b7dc-4e16-b410-4208b1b241c0 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "25d64f9f-18a6-4b6a-92b1-3e8e7d017ab8", "name": "network-vif-plugged", "status": "completed", "tag": "720bfaf8-2ab1-4a23-bccc-396cef466b3a"}]} {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 19 00:43:02.217323 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-5b00f052-f764-48f0-82e5-2a98356e7458 req-36ad9df6-b7dc-4e16-b410-4208b1b241c0 service nova] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:43:02.217895 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-5b00f052-f764-48f0-82e5-2a98356e7458 req-36ad9df6-b7dc-4e16-b410-4208b1b241c0 service nova] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:43:02.218400 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-5b00f052-f764-48f0-82e5-2a98356e7458 req-36ad9df6-b7dc-4e16-b410-4208b1b241c0 service nova] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:43:02.234605 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.compute.server_external_events [req-5b00f052-f764-48f0-82e5-2a98356e7458 req-36ad9df6-b7dc-4e16-b410-4208b1b241c0 service nova] Creating event network-vif-plugged:720bfaf8-2ab1-4a23-bccc-396cef466b3a for instance 25d64f9f-18a6-4b6a-92b1-3e8e7d017ab8 on hv-cinder-maste Oct 19 00:43:02.240990 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [req-5b00f052-f764-48f0-82e5-2a98356e7458 req-36ad9df6-b7dc-4e16-b410-4208b1b241c0 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.051456 Oct 19 00:43:02.241604 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 185/300] 10.222.0.26 () {64 vars in 1317 bytes} [Wed Oct 19 00:43:02 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 19 00:43:02.246892 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-c94f35c4-ed39-47bb-b45f-a8fd5c400b5a tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:43:02.248939 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-c94f35c4-ed39-47bb-b45f-a8fd5c400b5a tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] [instance: 25d64f9f-18a6-4b6a-92b1-3e8e7d017ab8] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:43:02.255735 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-c94f35c4-ed39-47bb-b45f-a8fd5c400b5a tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:43:02.256153 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-c94f35c4-ed39-47bb-b45f-a8fd5c400b5a tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:43:02.256578 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-c94f35c4-ed39-47bb-b45f-a8fd5c400b5a tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:43:02.257281 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-c94f35c4-ed39-47bb-b45f-a8fd5c400b5a tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:43:02.258031 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-c94f35c4-ed39-47bb-b45f-a8fd5c400b5a tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:43:02.258466 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-c94f35c4-ed39-47bb-b45f-a8fd5c400b5a tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:43:02.715734 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-c94f35c4-ed39-47bb-b45f-a8fd5c400b5a tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f8725c4d05b4dd79b05b43fba15980b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0b35c2038f6b4d0a9012b004f1416c00', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:43:02.727160 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-c94f35c4-ed39-47bb-b45f-a8fd5c400b5a tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] 10.222.0.26 "GET /compute/v2.1/servers/25d64f9f-18a6-4b6a-92b1-3e8e7d017ab8" status: 200 len: 1367 microversion: 2.1 time: 0.484323 Oct 19 00:43:02.742979 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 186/301] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:43:02 2022] GET /compute/v2.1/servers/25d64f9f-18a6-4b6a-92b1-3e8e7d017ab8 => generated 1367 bytes in 501 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:43:02.751000 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-93737a18-060b-4090-805e-46560c80498d tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:43:02.752657 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-93737a18-060b-4090-805e-46560c80498d tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] [instance: f6051269-1d16-42db-8323-67a532dbcf40] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:43:02.759899 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-93737a18-060b-4090-805e-46560c80498d tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:43:02.760762 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-93737a18-060b-4090-805e-46560c80498d tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:43:02.761469 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-93737a18-060b-4090-805e-46560c80498d tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:43:02.762753 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-93737a18-060b-4090-805e-46560c80498d tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:43:02.764371 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-93737a18-060b-4090-805e-46560c80498d tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:43:02.765083 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-93737a18-060b-4090-805e-46560c80498d tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:43:03.056551 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-93737a18-060b-4090-805e-46560c80498d tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd3f0d615187d4779bdd7da8c9f24c796', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0151d58abd047a692f5cc4e7d05bea5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:43:03.075892 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-93737a18-060b-4090-805e-46560c80498d tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] 10.222.0.26 "GET /compute/v2.1/servers/f6051269-1d16-42db-8323-67a532dbcf40" status: 200 len: 1372 microversion: 2.1 time: 0.331303 Oct 19 00:43:03.085289 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 187/302] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:43:02 2022] GET /compute/v2.1/servers/f6051269-1d16-42db-8323-67a532dbcf40 => generated 1372 bytes in 342 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:43:03.096360 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [req-58d74282-a19a-4f77-bd29-06a167c5f970 req-63bc806e-42e8-4c37-a48f-14e6d6652163 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "f6051269-1d16-42db-8323-67a532dbcf40", "tag": "667c66be-7422-49b6-8e96-6c36f067b697"}]} {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 19 00:43:03.114907 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-58d74282-a19a-4f77-bd29-06a167c5f970 req-63bc806e-42e8-4c37-a48f-14e6d6652163 service nova] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:43:03.115398 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-58d74282-a19a-4f77-bd29-06a167c5f970 req-63bc806e-42e8-4c37-a48f-14e6d6652163 service nova] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:43:03.115839 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-58d74282-a19a-4f77-bd29-06a167c5f970 req-63bc806e-42e8-4c37-a48f-14e6d6652163 service nova] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:43:03.130055 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.compute.server_external_events [req-58d74282-a19a-4f77-bd29-06a167c5f970 req-63bc806e-42e8-4c37-a48f-14e6d6652163 service nova] Creating event network-changed:667c66be-7422-49b6-8e96-6c36f067b697 for instance f6051269-1d16-42db-8323-67a532dbcf40 on hv-cinder-maste Oct 19 00:43:03.135750 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [req-58d74282-a19a-4f77-bd29-06a167c5f970 req-63bc806e-42e8-4c37-a48f-14e6d6652163 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.048366 Oct 19 00:43:03.136850 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 188/303] 10.222.0.26 () {64 vars in 1317 bytes} [Wed Oct 19 00:43:03 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 19 00:43:03.144480 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-d2bf5299-c068-4215-97a7-c45f7b470e06 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:43:03.145365 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-d2bf5299-c068-4215-97a7-c45f7b470e06 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] [instance: b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:43:03.152298 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d2bf5299-c068-4215-97a7-c45f7b470e06 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:43:03.153096 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d2bf5299-c068-4215-97a7-c45f7b470e06 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:43:03.154362 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d2bf5299-c068-4215-97a7-c45f7b470e06 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:43:03.155776 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d2bf5299-c068-4215-97a7-c45f7b470e06 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:43:03.156436 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d2bf5299-c068-4215-97a7-c45f7b470e06 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:43:03.157037 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d2bf5299-c068-4215-97a7-c45f7b470e06 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:43:03.333650 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-d2bf5299-c068-4215-97a7-c45f7b470e06 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:43:03.344843 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-d2bf5299-c068-4215-97a7-c45f7b470e06 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:43:03.346603 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-d2bf5299-c068-4215-97a7-c45f7b470e06 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:43:03.347817 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-d2bf5299-c068-4215-97a7-c45f7b470e06 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] 10.222.0.26 "GET /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2" status: 200 len: 1739 microversion: 2.20 time: 0.209815 Oct 19 00:43:03.348467 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 189/304] 10.222.0.26 () {60 vars in 1277 bytes} [Wed Oct 19 00:43:03 2022] GET /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2 => generated 1739 bytes in 211 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 19 00:43:03.355381 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-545b1646-0f19-409a-969d-c1045b47c19d tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:43:03.356289 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-545b1646-0f19-409a-969d-c1045b47c19d tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] [instance: c5d02d64-d4e6-445d-a155-b875b8a8cce0] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:43:03.375822 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-545b1646-0f19-409a-969d-c1045b47c19d tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:43:03.376244 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-545b1646-0f19-409a-969d-c1045b47c19d tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:43:03.377061 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-545b1646-0f19-409a-969d-c1045b47c19d tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:43:03.377859 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-545b1646-0f19-409a-969d-c1045b47c19d tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:43:03.378408 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-545b1646-0f19-409a-969d-c1045b47c19d tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:43:03.378863 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-545b1646-0f19-409a-969d-c1045b47c19d tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:43:03.469435 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-545b1646-0f19-409a-969d-c1045b47c19d tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a08c9457231a447eb1c4130e68cfb31d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '173a52b79dab4962a4aa380c26f97d2d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:43:03.481351 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-545b1646-0f19-409a-969d-c1045b47c19d tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] 10.222.0.26 "GET /compute/v2.1/servers/c5d02d64-d4e6-445d-a155-b875b8a8cce0" status: 200 len: 1461 microversion: 2.1 time: 0.131664 Oct 19 00:43:03.482793 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 190/305] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:43:03 2022] GET /compute/v2.1/servers/c5d02d64-d4e6-445d-a155-b875b8a8cce0 => generated 1461 bytes in 134 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:43:03.489046 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-4a32e923-06ed-4ed6-8b80-2ab7f24a8173 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:43:03.490081 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-4a32e923-06ed-4ed6-8b80-2ab7f24a8173 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] [instance: 5bdc36c3-1db5-4ff2-808d-1295f314d522] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:43:03.500194 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-4a32e923-06ed-4ed6-8b80-2ab7f24a8173 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:43:03.501154 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-4a32e923-06ed-4ed6-8b80-2ab7f24a8173 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:43:03.501728 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-4a32e923-06ed-4ed6-8b80-2ab7f24a8173 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:43:03.502437 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-4a32e923-06ed-4ed6-8b80-2ab7f24a8173 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:43:03.503109 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-4a32e923-06ed-4ed6-8b80-2ab7f24a8173 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:43:03.503558 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-4a32e923-06ed-4ed6-8b80-2ab7f24a8173 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:43:03.597568 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-4a32e923-06ed-4ed6-8b80-2ab7f24a8173 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '39de6a6fff7c4cc08cc62aa86182d71d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1235d8b054b3405595fca761f08b73ae', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:43:03.608948 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-4a32e923-06ed-4ed6-8b80-2ab7f24a8173 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] 10.222.0.26 "GET /compute/v2.1/servers/5bdc36c3-1db5-4ff2-808d-1295f314d522" status: 200 len: 1277 microversion: 2.1 time: 0.124900 Oct 19 00:43:03.609530 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 191/306] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:43:03 2022] GET /compute/v2.1/servers/5bdc36c3-1db5-4ff2-808d-1295f314d522 => generated 1277 bytes in 126 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:43:03.680212 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [req-a06afa80-2233-437d-89e7-e3350ab53104 req-b534cd7e-576b-48d9-bb4d-b9e414bf3178 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "c5d02d64-d4e6-445d-a155-b875b8a8cce0", "name": "network-vif-deleted", "tag": "1b4d823f-e9a3-4a43-a759-c283b8c3dac4"}]} {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 19 00:43:03.697380 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-a06afa80-2233-437d-89e7-e3350ab53104 req-b534cd7e-576b-48d9-bb4d-b9e414bf3178 service nova] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:43:03.698199 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-a06afa80-2233-437d-89e7-e3350ab53104 req-b534cd7e-576b-48d9-bb4d-b9e414bf3178 service nova] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:43:03.699123 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-a06afa80-2233-437d-89e7-e3350ab53104 req-b534cd7e-576b-48d9-bb4d-b9e414bf3178 service nova] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:43:03.718238 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.compute.server_external_events [req-a06afa80-2233-437d-89e7-e3350ab53104 req-b534cd7e-576b-48d9-bb4d-b9e414bf3178 service nova] Creating event network-vif-deleted:1b4d823f-e9a3-4a43-a759-c283b8c3dac4 for instance c5d02d64-d4e6-445d-a155-b875b8a8cce0 on hv-cinder-maste Oct 19 00:43:03.726997 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [req-a06afa80-2233-437d-89e7-e3350ab53104 req-b534cd7e-576b-48d9-bb4d-b9e414bf3178 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.051004 Oct 19 00:43:03.728615 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 192/307] 10.222.0.26 () {64 vars in 1317 bytes} [Wed Oct 19 00:43:03 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 19 00:43:03.770280 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-090bc2ee-f2b0-4245-895a-30a0ff3299a7 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:43:03.774200 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-090bc2ee-f2b0-4245-895a-30a0ff3299a7 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] [instance: 25d64f9f-18a6-4b6a-92b1-3e8e7d017ab8] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:43:03.784163 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-090bc2ee-f2b0-4245-895a-30a0ff3299a7 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:43:03.784660 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-090bc2ee-f2b0-4245-895a-30a0ff3299a7 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:43:03.785247 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-090bc2ee-f2b0-4245-895a-30a0ff3299a7 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:43:03.785975 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-090bc2ee-f2b0-4245-895a-30a0ff3299a7 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:43:03.786427 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-090bc2ee-f2b0-4245-895a-30a0ff3299a7 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:43:03.787623 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-090bc2ee-f2b0-4245-895a-30a0ff3299a7 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:43:03.962509 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-090bc2ee-f2b0-4245-895a-30a0ff3299a7 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f8725c4d05b4dd79b05b43fba15980b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0b35c2038f6b4d0a9012b004f1416c00', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:43:03.972507 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-090bc2ee-f2b0-4245-895a-30a0ff3299a7 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] 10.222.0.26 "GET /compute/v2.1/servers/25d64f9f-18a6-4b6a-92b1-3e8e7d017ab8" status: 200 len: 1367 microversion: 2.1 time: 0.217921 Oct 19 00:43:03.973431 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 193/308] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:43:03 2022] GET /compute/v2.1/servers/25d64f9f-18a6-4b6a-92b1-3e8e7d017ab8 => generated 1367 bytes in 220 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:43:04.147494 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-7051b9d8-ae4d-4d2f-940a-b06cc75c0d7c tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:43:04.148213 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-7051b9d8-ae4d-4d2f-940a-b06cc75c0d7c tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] [instance: f6051269-1d16-42db-8323-67a532dbcf40] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:43:04.165716 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-7051b9d8-ae4d-4d2f-940a-b06cc75c0d7c tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:43:04.166223 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-7051b9d8-ae4d-4d2f-940a-b06cc75c0d7c tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:43:04.166716 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-7051b9d8-ae4d-4d2f-940a-b06cc75c0d7c tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:43:04.167446 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-7051b9d8-ae4d-4d2f-940a-b06cc75c0d7c tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:43:04.167909 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-7051b9d8-ae4d-4d2f-940a-b06cc75c0d7c tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:43:04.168392 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-7051b9d8-ae4d-4d2f-940a-b06cc75c0d7c tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:43:04.187118 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG dbcounter [-] [125641] Writing DB stats nova_cell1:SELECT=100 {{(pid=125641) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 00:43:04.427586 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-7051b9d8-ae4d-4d2f-940a-b06cc75c0d7c tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd3f0d615187d4779bdd7da8c9f24c796', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0151d58abd047a692f5cc4e7d05bea5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:43:04.437354 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-7051b9d8-ae4d-4d2f-940a-b06cc75c0d7c tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] 10.222.0.26 "GET /compute/v2.1/servers/f6051269-1d16-42db-8323-67a532dbcf40" status: 200 len: 1372 microversion: 2.1 time: 0.326020 Oct 19 00:43:04.438825 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 194/309] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:43:04 2022] GET /compute/v2.1/servers/f6051269-1d16-42db-8323-67a532dbcf40 => generated 1372 bytes in 328 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 19 00:43:04.444080 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-266606fd-5c86-40a8-96b0-42310d3c2662 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:43:04.444888 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-266606fd-5c86-40a8-96b0-42310d3c2662 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] [instance: b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:43:04.461820 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-266606fd-5c86-40a8-96b0-42310d3c2662 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:43:04.462350 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-266606fd-5c86-40a8-96b0-42310d3c2662 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:43:04.462765 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-266606fd-5c86-40a8-96b0-42310d3c2662 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:43:04.463474 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-266606fd-5c86-40a8-96b0-42310d3c2662 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:43:04.465291 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-266606fd-5c86-40a8-96b0-42310d3c2662 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:43:04.465798 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-266606fd-5c86-40a8-96b0-42310d3c2662 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:43:04.679224 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-266606fd-5c86-40a8-96b0-42310d3c2662 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:43:04.690376 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-266606fd-5c86-40a8-96b0-42310d3c2662 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:43:04.693070 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-266606fd-5c86-40a8-96b0-42310d3c2662 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:43:04.696258 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-266606fd-5c86-40a8-96b0-42310d3c2662 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] 10.222.0.26 "GET /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2" status: 200 len: 1739 microversion: 2.20 time: 0.256495 Oct 19 00:43:04.697774 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 195/310] 10.222.0.26 () {60 vars in 1277 bytes} [Wed Oct 19 00:43:04 2022] GET /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2 => generated 1739 bytes in 259 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 19 00:43:04.704191 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-9fa88e03-6654-4382-91fc-ece1e9b92bf3 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:43:04.705230 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-9fa88e03-6654-4382-91fc-ece1e9b92bf3 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] [instance: c5d02d64-d4e6-445d-a155-b875b8a8cce0] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:43:04.713215 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-9fa88e03-6654-4382-91fc-ece1e9b92bf3 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:43:04.713994 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-9fa88e03-6654-4382-91fc-ece1e9b92bf3 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:43:04.714661 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-9fa88e03-6654-4382-91fc-ece1e9b92bf3 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:43:04.715692 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-9fa88e03-6654-4382-91fc-ece1e9b92bf3 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:43:04.716776 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-9fa88e03-6654-4382-91fc-ece1e9b92bf3 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:43:04.717449 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-9fa88e03-6654-4382-91fc-ece1e9b92bf3 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:43:04.757312 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-9fa88e03-6654-4382-91fc-ece1e9b92bf3 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:43:04.758679 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-9fa88e03-6654-4382-91fc-ece1e9b92bf3 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:43:04.759321 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-9fa88e03-6654-4382-91fc-ece1e9b92bf3 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:43:04.759993 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.objects.instance [None req-9fa88e03-6654-4382-91fc-ece1e9b92bf3 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Lazy-loading 'fault' on Instance uuid c5d02d64-d4e6-445d-a155-b875b8a8cce0 {{(pid=125641) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 19 00:43:04.809086 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-9fa88e03-6654-4382-91fc-ece1e9b92bf3 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a08c9457231a447eb1c4130e68cfb31d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '173a52b79dab4962a4aa380c26f97d2d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:43:04.821159 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-9fa88e03-6654-4382-91fc-ece1e9b92bf3 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] 10.222.0.26 "GET /compute/v2.1/servers/c5d02d64-d4e6-445d-a155-b875b8a8cce0" status: 200 len: 1304 microversion: 2.1 time: 0.121365 Oct 19 00:43:04.821673 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 196/311] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:43:04 2022] GET /compute/v2.1/servers/c5d02d64-d4e6-445d-a155-b875b8a8cce0 => generated 1304 bytes in 123 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:43:04.829881 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-29f55d2b-cbbc-4c34-adb5-7e4c41479936 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:43:04.830775 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-29f55d2b-cbbc-4c34-adb5-7e4c41479936 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] [instance: 5bdc36c3-1db5-4ff2-808d-1295f314d522] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:43:04.848192 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-29f55d2b-cbbc-4c34-adb5-7e4c41479936 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:43:04.848956 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-29f55d2b-cbbc-4c34-adb5-7e4c41479936 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:43:04.849583 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-29f55d2b-cbbc-4c34-adb5-7e4c41479936 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:43:04.850472 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-29f55d2b-cbbc-4c34-adb5-7e4c41479936 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:43:04.851114 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-29f55d2b-cbbc-4c34-adb5-7e4c41479936 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:43:04.851705 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-29f55d2b-cbbc-4c34-adb5-7e4c41479936 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:43:05.053103 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-29f55d2b-cbbc-4c34-adb5-7e4c41479936 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '39de6a6fff7c4cc08cc62aa86182d71d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1235d8b054b3405595fca761f08b73ae', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:43:05.063764 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-29f55d2b-cbbc-4c34-adb5-7e4c41479936 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] 10.222.0.26 "GET /compute/v2.1/servers/5bdc36c3-1db5-4ff2-808d-1295f314d522" status: 200 len: 1319 microversion: 2.1 time: 0.240903 Oct 19 00:43:05.064664 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 197/312] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:43:04 2022] GET /compute/v2.1/servers/5bdc36c3-1db5-4ff2-808d-1295f314d522 => generated 1319 bytes in 242 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:43:05.075812 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-7bac62de-66a4-4b03-aea1-04033041a677 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:43:05.078320 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-7bac62de-66a4-4b03-aea1-04033041a677 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] [instance: 25d64f9f-18a6-4b6a-92b1-3e8e7d017ab8] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:43:05.088174 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-7bac62de-66a4-4b03-aea1-04033041a677 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:43:05.090286 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-7bac62de-66a4-4b03-aea1-04033041a677 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:43:05.092471 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-7bac62de-66a4-4b03-aea1-04033041a677 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:43:05.093576 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-7bac62de-66a4-4b03-aea1-04033041a677 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:43:05.094056 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-7bac62de-66a4-4b03-aea1-04033041a677 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:43:05.094517 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-7bac62de-66a4-4b03-aea1-04033041a677 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:43:05.313705 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-7bac62de-66a4-4b03-aea1-04033041a677 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f8725c4d05b4dd79b05b43fba15980b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0b35c2038f6b4d0a9012b004f1416c00', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:43:05.323303 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-7bac62de-66a4-4b03-aea1-04033041a677 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] 10.222.0.26 "GET /compute/v2.1/servers/25d64f9f-18a6-4b6a-92b1-3e8e7d017ab8" status: 200 len: 1367 microversion: 2.1 time: 0.255558 Oct 19 00:43:05.339305 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 198/313] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:43:05 2022] GET /compute/v2.1/servers/25d64f9f-18a6-4b6a-92b1-3e8e7d017ab8 => generated 1367 bytes in 260 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:43:05.464704 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-a9abd111-29f9-4931-b82e-e39ff3ee7535 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:43:05.465629 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-a9abd111-29f9-4931-b82e-e39ff3ee7535 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] [instance: f6051269-1d16-42db-8323-67a532dbcf40] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:43:05.474020 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-a9abd111-29f9-4931-b82e-e39ff3ee7535 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:43:05.474527 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-a9abd111-29f9-4931-b82e-e39ff3ee7535 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:43:05.475166 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-a9abd111-29f9-4931-b82e-e39ff3ee7535 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:43:05.476047 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-a9abd111-29f9-4931-b82e-e39ff3ee7535 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:43:05.476677 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-a9abd111-29f9-4931-b82e-e39ff3ee7535 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:43:05.477274 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-a9abd111-29f9-4931-b82e-e39ff3ee7535 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:43:05.645168 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-a9abd111-29f9-4931-b82e-e39ff3ee7535 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd3f0d615187d4779bdd7da8c9f24c796', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0151d58abd047a692f5cc4e7d05bea5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:43:05.653649 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-a9abd111-29f9-4931-b82e-e39ff3ee7535 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] 10.222.0.26 "GET /compute/v2.1/servers/f6051269-1d16-42db-8323-67a532dbcf40" status: 200 len: 1372 microversion: 2.1 time: 0.192953 Oct 19 00:43:05.655001 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 199/314] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:43:05 2022] GET /compute/v2.1/servers/f6051269-1d16-42db-8323-67a532dbcf40 => generated 1372 bytes in 195 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:43:05.724219 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-c2259789-b42a-48db-a5f5-4446d5c0e413 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:43:05.724828 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-c2259789-b42a-48db-a5f5-4446d5c0e413 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] [instance: b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:43:05.731759 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-c2259789-b42a-48db-a5f5-4446d5c0e413 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:43:05.732380 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-c2259789-b42a-48db-a5f5-4446d5c0e413 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:43:05.732786 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-c2259789-b42a-48db-a5f5-4446d5c0e413 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:43:05.733484 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-c2259789-b42a-48db-a5f5-4446d5c0e413 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:43:05.733934 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-c2259789-b42a-48db-a5f5-4446d5c0e413 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:43:05.734368 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-c2259789-b42a-48db-a5f5-4446d5c0e413 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:43:05.791917 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG dbcounter [-] [125641] Writing DB stats nova_cell0:SELECT=6 {{(pid=125641) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 00:43:05.937457 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-c2259789-b42a-48db-a5f5-4446d5c0e413 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:43:05.949556 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-c2259789-b42a-48db-a5f5-4446d5c0e413 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:43:05.949556 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-c2259789-b42a-48db-a5f5-4446d5c0e413 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:43:05.950353 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-c2259789-b42a-48db-a5f5-4446d5c0e413 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] 10.222.0.26 "GET /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2" status: 200 len: 1739 microversion: 2.20 time: 0.230304 Oct 19 00:43:05.958605 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 200/315] 10.222.0.26 () {60 vars in 1277 bytes} [Wed Oct 19 00:43:05 2022] GET /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2 => generated 1739 bytes in 239 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 19 00:43:05.963588 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-ca6f700b-6db1-4fb4-8ee0-51a3ef75ae5f tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:43:05.964471 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-ca6f700b-6db1-4fb4-8ee0-51a3ef75ae5f tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] [instance: c5d02d64-d4e6-445d-a155-b875b8a8cce0] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:43:05.974402 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ca6f700b-6db1-4fb4-8ee0-51a3ef75ae5f tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:43:05.975212 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ca6f700b-6db1-4fb4-8ee0-51a3ef75ae5f tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:43:05.975832 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ca6f700b-6db1-4fb4-8ee0-51a3ef75ae5f tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:43:05.977039 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ca6f700b-6db1-4fb4-8ee0-51a3ef75ae5f tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:43:05.991244 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ca6f700b-6db1-4fb4-8ee0-51a3ef75ae5f tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:43:05.991244 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ca6f700b-6db1-4fb4-8ee0-51a3ef75ae5f tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:43:06.015793 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ca6f700b-6db1-4fb4-8ee0-51a3ef75ae5f tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:43:06.016274 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ca6f700b-6db1-4fb4-8ee0-51a3ef75ae5f tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:43:06.016705 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ca6f700b-6db1-4fb4-8ee0-51a3ef75ae5f tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:43:06.017147 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.objects.instance [None req-ca6f700b-6db1-4fb4-8ee0-51a3ef75ae5f tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Lazy-loading 'fault' on Instance uuid c5d02d64-d4e6-445d-a155-b875b8a8cce0 {{(pid=125641) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 19 00:43:06.067546 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-ca6f700b-6db1-4fb4-8ee0-51a3ef75ae5f tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a08c9457231a447eb1c4130e68cfb31d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '173a52b79dab4962a4aa380c26f97d2d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:43:06.121897 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-ca6f700b-6db1-4fb4-8ee0-51a3ef75ae5f tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] 10.222.0.26 "GET /compute/v2.1/servers/c5d02d64-d4e6-445d-a155-b875b8a8cce0" status: 200 len: 1304 microversion: 2.1 time: 0.149036 Oct 19 00:43:06.121897 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 201/316] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:43:05 2022] GET /compute/v2.1/servers/c5d02d64-d4e6-445d-a155-b875b8a8cce0 => generated 1304 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:43:06.139744 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-f484a83b-859c-44d9-95c9-16aee954f2bb tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:43:06.140372 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-f484a83b-859c-44d9-95c9-16aee954f2bb tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] [instance: 5bdc36c3-1db5-4ff2-808d-1295f314d522] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:43:06.151624 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-f484a83b-859c-44d9-95c9-16aee954f2bb tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:43:06.152007 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-f484a83b-859c-44d9-95c9-16aee954f2bb tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:43:06.152446 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-f484a83b-859c-44d9-95c9-16aee954f2bb tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:43:06.158669 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-f484a83b-859c-44d9-95c9-16aee954f2bb tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:43:06.159155 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-f484a83b-859c-44d9-95c9-16aee954f2bb tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:43:06.159592 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-f484a83b-859c-44d9-95c9-16aee954f2bb tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:43:06.376625 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-f484a83b-859c-44d9-95c9-16aee954f2bb tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '39de6a6fff7c4cc08cc62aa86182d71d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1235d8b054b3405595fca761f08b73ae', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:43:06.386835 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-f484a83b-859c-44d9-95c9-16aee954f2bb tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] 10.222.0.26 "GET /compute/v2.1/servers/5bdc36c3-1db5-4ff2-808d-1295f314d522" status: 200 len: 1319 microversion: 2.1 time: 0.271051 Oct 19 00:43:06.402087 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 202/317] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:43:06 2022] GET /compute/v2.1/servers/5bdc36c3-1db5-4ff2-808d-1295f314d522 => generated 1319 bytes in 275 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:43:06.402087 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-015b3699-fa0a-4e32-b22a-9d583656f619 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:43:06.402087 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-015b3699-fa0a-4e32-b22a-9d583656f619 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] [instance: 25d64f9f-18a6-4b6a-92b1-3e8e7d017ab8] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:43:06.402087 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG dbcounter [-] [125641] Writing DB stats nova_api:SELECT=99,nova_api:UPDATE=1 {{(pid=125641) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 00:43:06.408923 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-015b3699-fa0a-4e32-b22a-9d583656f619 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:43:06.409556 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-015b3699-fa0a-4e32-b22a-9d583656f619 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:43:06.410094 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-015b3699-fa0a-4e32-b22a-9d583656f619 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:43:06.411108 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-015b3699-fa0a-4e32-b22a-9d583656f619 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:43:06.411667 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-015b3699-fa0a-4e32-b22a-9d583656f619 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:43:06.412213 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-015b3699-fa0a-4e32-b22a-9d583656f619 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:43:06.585588 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-015b3699-fa0a-4e32-b22a-9d583656f619 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f8725c4d05b4dd79b05b43fba15980b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0b35c2038f6b4d0a9012b004f1416c00', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:43:06.596375 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-015b3699-fa0a-4e32-b22a-9d583656f619 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] 10.222.0.26 "GET /compute/v2.1/servers/25d64f9f-18a6-4b6a-92b1-3e8e7d017ab8" status: 200 len: 1556 microversion: 2.1 time: 0.206977 Oct 19 00:43:06.597251 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 203/318] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:43:06 2022] GET /compute/v2.1/servers/25d64f9f-18a6-4b6a-92b1-3e8e7d017ab8 => generated 1556 bytes in 209 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:43:06.626037 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-3914a252-24a8-4654-921a-9488783ad571 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:43:06.627153 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-3914a252-24a8-4654-921a-9488783ad571 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] [instance: 25d64f9f-18a6-4b6a-92b1-3e8e7d017ab8] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:43:06.629463 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG dbcounter [-] [125641] Writing DB stats nova_cell0:SELECT=38 {{(pid=125641) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 00:43:06.635762 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-3914a252-24a8-4654-921a-9488783ad571 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:43:06.636256 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-3914a252-24a8-4654-921a-9488783ad571 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:43:06.639697 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-3914a252-24a8-4654-921a-9488783ad571 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:43:06.641723 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-3914a252-24a8-4654-921a-9488783ad571 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:43:06.643034 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-3914a252-24a8-4654-921a-9488783ad571 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:43:06.643526 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-3914a252-24a8-4654-921a-9488783ad571 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:43:06.765471 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-3914a252-24a8-4654-921a-9488783ad571 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] 10.222.0.26 "GET /compute/v2.1/servers/25d64f9f-18a6-4b6a-92b1-3e8e7d017ab8/os-interface" status: 200 len: 285 microversion: 2.1 time: 0.144025 Oct 19 00:43:06.773759 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 204/319] 10.222.0.26 () {58 vars in 1275 bytes} [Wed Oct 19 00:43:06 2022] GET /compute/v2.1/servers/25d64f9f-18a6-4b6a-92b1-3e8e7d017ab8/os-interface => generated 285 bytes in 146 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 19 00:43:06.773759 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-4908d98c-3736-425d-b861-c49c76bda57f tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:43:06.773759 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-4908d98c-3736-425d-b861-c49c76bda57f tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] [instance: f6051269-1d16-42db-8323-67a532dbcf40] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:43:06.785915 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-4908d98c-3736-425d-b861-c49c76bda57f tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:43:06.786517 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-4908d98c-3736-425d-b861-c49c76bda57f tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:43:06.787054 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-4908d98c-3736-425d-b861-c49c76bda57f tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:43:06.787857 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-4908d98c-3736-425d-b861-c49c76bda57f tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:43:06.788736 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-4908d98c-3736-425d-b861-c49c76bda57f tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:43:06.789177 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-4908d98c-3736-425d-b861-c49c76bda57f tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:43:06.940464 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-4908d98c-3736-425d-b861-c49c76bda57f tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd3f0d615187d4779bdd7da8c9f24c796', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0151d58abd047a692f5cc4e7d05bea5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:43:06.948783 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-4908d98c-3736-425d-b861-c49c76bda57f tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] 10.222.0.26 "GET /compute/v2.1/servers/f6051269-1d16-42db-8323-67a532dbcf40" status: 200 len: 1372 microversion: 2.1 time: 0.181149 Oct 19 00:43:06.949678 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 205/320] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:43:06 2022] GET /compute/v2.1/servers/f6051269-1d16-42db-8323-67a532dbcf40 => generated 1372 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:43:06.977298 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-cb7a64bb-d22c-44a2-9409-740627fa2550 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:43:06.977915 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-cb7a64bb-d22c-44a2-9409-740627fa2550 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] [instance: b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:43:06.984761 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-cb7a64bb-d22c-44a2-9409-740627fa2550 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:43:06.985255 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-cb7a64bb-d22c-44a2-9409-740627fa2550 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:43:06.985693 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-cb7a64bb-d22c-44a2-9409-740627fa2550 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:43:06.986399 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-cb7a64bb-d22c-44a2-9409-740627fa2550 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:43:06.986907 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-cb7a64bb-d22c-44a2-9409-740627fa2550 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:43:06.987332 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-cb7a64bb-d22c-44a2-9409-740627fa2550 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:43:07.154750 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-cb7a64bb-d22c-44a2-9409-740627fa2550 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:43:07.164226 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-cb7a64bb-d22c-44a2-9409-740627fa2550 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:43:07.165873 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-cb7a64bb-d22c-44a2-9409-740627fa2550 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:43:07.167457 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-cb7a64bb-d22c-44a2-9409-740627fa2550 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] 10.222.0.26 "GET /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2" status: 200 len: 1739 microversion: 2.20 time: 0.194177 Oct 19 00:43:07.168589 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 206/321] 10.222.0.26 () {60 vars in 1277 bytes} [Wed Oct 19 00:43:06 2022] GET /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2 => generated 1739 bytes in 196 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 19 00:43:07.174421 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-7991a28d-83ef-4567-8a97-f4cfe967efa1 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:43:07.175070 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-7991a28d-83ef-4567-8a97-f4cfe967efa1 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] [instance: c5d02d64-d4e6-445d-a155-b875b8a8cce0] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:43:07.181430 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-7991a28d-83ef-4567-8a97-f4cfe967efa1 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:43:07.181915 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-7991a28d-83ef-4567-8a97-f4cfe967efa1 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:43:07.182354 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-7991a28d-83ef-4567-8a97-f4cfe967efa1 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:43:07.183081 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-7991a28d-83ef-4567-8a97-f4cfe967efa1 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:43:07.183537 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-7991a28d-83ef-4567-8a97-f4cfe967efa1 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:43:07.183959 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-7991a28d-83ef-4567-8a97-f4cfe967efa1 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:43:07.220626 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-7991a28d-83ef-4567-8a97-f4cfe967efa1 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:43:07.221108 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-7991a28d-83ef-4567-8a97-f4cfe967efa1 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:43:07.221392 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-7991a28d-83ef-4567-8a97-f4cfe967efa1 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:43:07.221845 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.objects.instance [None req-7991a28d-83ef-4567-8a97-f4cfe967efa1 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Lazy-loading 'fault' on Instance uuid c5d02d64-d4e6-445d-a155-b875b8a8cce0 {{(pid=125641) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 19 00:43:07.265914 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-7991a28d-83ef-4567-8a97-f4cfe967efa1 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a08c9457231a447eb1c4130e68cfb31d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '173a52b79dab4962a4aa380c26f97d2d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:43:07.274573 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-7991a28d-83ef-4567-8a97-f4cfe967efa1 tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] 10.222.0.26 "GET /compute/v2.1/servers/c5d02d64-d4e6-445d-a155-b875b8a8cce0" status: 200 len: 1304 microversion: 2.1 time: 0.103969 Oct 19 00:43:07.275162 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 207/322] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:43:07 2022] GET /compute/v2.1/servers/c5d02d64-d4e6-445d-a155-b875b8a8cce0 => generated 1304 bytes in 106 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:43:07.416616 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-8b322c0b-79a6-476c-9c05-b03613be3680 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:43:07.417201 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-8b322c0b-79a6-476c-9c05-b03613be3680 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] [instance: 5bdc36c3-1db5-4ff2-808d-1295f314d522] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:43:07.424989 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-8b322c0b-79a6-476c-9c05-b03613be3680 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:43:07.425598 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-8b322c0b-79a6-476c-9c05-b03613be3680 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:43:07.426272 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-8b322c0b-79a6-476c-9c05-b03613be3680 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:43:07.427107 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-8b322c0b-79a6-476c-9c05-b03613be3680 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:43:07.427637 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-8b322c0b-79a6-476c-9c05-b03613be3680 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:43:07.428450 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-8b322c0b-79a6-476c-9c05-b03613be3680 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:43:07.608632 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-8b322c0b-79a6-476c-9c05-b03613be3680 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '39de6a6fff7c4cc08cc62aa86182d71d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1235d8b054b3405595fca761f08b73ae', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:43:07.622748 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-8b322c0b-79a6-476c-9c05-b03613be3680 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] 10.222.0.26 "GET /compute/v2.1/servers/5bdc36c3-1db5-4ff2-808d-1295f314d522" status: 200 len: 1319 microversion: 2.1 time: 0.210429 Oct 19 00:43:07.639238 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 208/323] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:43:07 2022] GET /compute/v2.1/servers/5bdc36c3-1db5-4ff2-808d-1295f314d522 => generated 1319 bytes in 227 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:43:07.973995 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-5f577979-2fed-4041-81e3-edc7e7bd493d tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:43:07.974987 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-5f577979-2fed-4041-81e3-edc7e7bd493d tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] [instance: f6051269-1d16-42db-8323-67a532dbcf40] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:43:07.982983 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-5f577979-2fed-4041-81e3-edc7e7bd493d tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:43:07.983298 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-5f577979-2fed-4041-81e3-edc7e7bd493d tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:43:07.983739 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-5f577979-2fed-4041-81e3-edc7e7bd493d tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:43:07.984451 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-5f577979-2fed-4041-81e3-edc7e7bd493d tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:43:07.985089 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-5f577979-2fed-4041-81e3-edc7e7bd493d tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:43:07.985525 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-5f577979-2fed-4041-81e3-edc7e7bd493d tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:43:08.210993 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-5f577979-2fed-4041-81e3-edc7e7bd493d tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd3f0d615187d4779bdd7da8c9f24c796', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0151d58abd047a692f5cc4e7d05bea5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:43:08.229376 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-5f577979-2fed-4041-81e3-edc7e7bd493d tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] 10.222.0.26 "GET /compute/v2.1/servers/f6051269-1d16-42db-8323-67a532dbcf40" status: 200 len: 1372 microversion: 2.1 time: 0.254534 Oct 19 00:43:08.233221 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 209/324] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:43:07 2022] GET /compute/v2.1/servers/f6051269-1d16-42db-8323-67a532dbcf40 => generated 1372 bytes in 264 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:43:08.246873 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-ec6fcdd1-69c7-47b6-b6b1-008bcfb8b914 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:43:08.248374 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-ec6fcdd1-69c7-47b6-b6b1-008bcfb8b914 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] [instance: b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:43:08.255651 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ec6fcdd1-69c7-47b6-b6b1-008bcfb8b914 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:43:08.256190 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ec6fcdd1-69c7-47b6-b6b1-008bcfb8b914 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:43:08.256685 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ec6fcdd1-69c7-47b6-b6b1-008bcfb8b914 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:43:08.257499 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ec6fcdd1-69c7-47b6-b6b1-008bcfb8b914 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:43:08.258266 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ec6fcdd1-69c7-47b6-b6b1-008bcfb8b914 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:43:08.258745 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ec6fcdd1-69c7-47b6-b6b1-008bcfb8b914 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:43:08.265544 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG dbcounter [-] [125641] Writing DB stats nova_cell1:SELECT=100 {{(pid=125641) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 00:43:08.449285 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-ec6fcdd1-69c7-47b6-b6b1-008bcfb8b914 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:43:08.460345 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-ec6fcdd1-69c7-47b6-b6b1-008bcfb8b914 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:43:08.462151 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-ec6fcdd1-69c7-47b6-b6b1-008bcfb8b914 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:43:08.463441 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-ec6fcdd1-69c7-47b6-b6b1-008bcfb8b914 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] 10.222.0.26 "GET /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2" status: 200 len: 1739 microversion: 2.20 time: 0.228087 Oct 19 00:43:08.464494 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 210/325] 10.222.0.26 () {60 vars in 1277 bytes} [Wed Oct 19 00:43:08 2022] GET /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2 => generated 1739 bytes in 230 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 19 00:43:08.474948 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [req-ad4f5fc4-4afa-487a-b3e7-b0869061f09f req-9522046c-519e-403e-a08c-e832347286ce service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "25d64f9f-18a6-4b6a-92b1-3e8e7d017ab8", "tag": "720bfaf8-2ab1-4a23-bccc-396cef466b3a"}]} {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 19 00:43:08.491973 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-ad4f5fc4-4afa-487a-b3e7-b0869061f09f req-9522046c-519e-403e-a08c-e832347286ce service nova] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:43:08.492455 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-ad4f5fc4-4afa-487a-b3e7-b0869061f09f req-9522046c-519e-403e-a08c-e832347286ce service nova] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:43:08.492882 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-ad4f5fc4-4afa-487a-b3e7-b0869061f09f req-9522046c-519e-403e-a08c-e832347286ce service nova] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:43:08.508838 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.compute.server_external_events [req-ad4f5fc4-4afa-487a-b3e7-b0869061f09f req-9522046c-519e-403e-a08c-e832347286ce service nova] Creating event network-changed:720bfaf8-2ab1-4a23-bccc-396cef466b3a for instance 25d64f9f-18a6-4b6a-92b1-3e8e7d017ab8 on hv-cinder-maste Oct 19 00:43:08.515367 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [req-ad4f5fc4-4afa-487a-b3e7-b0869061f09f req-9522046c-519e-403e-a08c-e832347286ce service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.049528 Oct 19 00:43:08.516071 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 211/326] 10.222.0.26 () {64 vars in 1317 bytes} [Wed Oct 19 00:43:08 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 19 00:43:08.524420 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-59fcb395-55ad-4106-8d12-1454ca493a5f tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:43:08.525116 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-59fcb395-55ad-4106-8d12-1454ca493a5f tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] [instance: c5d02d64-d4e6-445d-a155-b875b8a8cce0] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:43:08.532882 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-59fcb395-55ad-4106-8d12-1454ca493a5f tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:43:08.533481 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-59fcb395-55ad-4106-8d12-1454ca493a5f tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:43:08.534157 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-59fcb395-55ad-4106-8d12-1454ca493a5f tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:43:08.535872 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-59fcb395-55ad-4106-8d12-1454ca493a5f tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:43:08.536391 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-59fcb395-55ad-4106-8d12-1454ca493a5f tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:43:08.537136 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-59fcb395-55ad-4106-8d12-1454ca493a5f tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:43:08.574532 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-59fcb395-55ad-4106-8d12-1454ca493a5f tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:43:08.575686 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-59fcb395-55ad-4106-8d12-1454ca493a5f tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:43:08.576934 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-59fcb395-55ad-4106-8d12-1454ca493a5f tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:43:08.577612 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.objects.instance [None req-59fcb395-55ad-4106-8d12-1454ca493a5f tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Lazy-loading 'fault' on Instance uuid c5d02d64-d4e6-445d-a155-b875b8a8cce0 {{(pid=125641) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 19 00:43:08.632141 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-59fcb395-55ad-4106-8d12-1454ca493a5f tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a08c9457231a447eb1c4130e68cfb31d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '173a52b79dab4962a4aa380c26f97d2d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:43:08.642901 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-59fcb395-55ad-4106-8d12-1454ca493a5f tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] 10.222.0.26 "GET /compute/v2.1/servers/c5d02d64-d4e6-445d-a155-b875b8a8cce0" status: 200 len: 1304 microversion: 2.1 time: 0.122298 Oct 19 00:43:08.643940 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 212/327] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:43:08 2022] GET /compute/v2.1/servers/c5d02d64-d4e6-445d-a155-b875b8a8cce0 => generated 1304 bytes in 124 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:43:08.665374 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-fa9210e4-0542-4fca-aa0e-b722fd089f11 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:43:08.671552 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-fa9210e4-0542-4fca-aa0e-b722fd089f11 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] [instance: 5bdc36c3-1db5-4ff2-808d-1295f314d522] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:43:08.704556 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-fa9210e4-0542-4fca-aa0e-b722fd089f11 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:43:08.705154 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-fa9210e4-0542-4fca-aa0e-b722fd089f11 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:43:08.705907 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-fa9210e4-0542-4fca-aa0e-b722fd089f11 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:43:08.708733 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-fa9210e4-0542-4fca-aa0e-b722fd089f11 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:43:08.709539 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-fa9210e4-0542-4fca-aa0e-b722fd089f11 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:43:08.710055 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-fa9210e4-0542-4fca-aa0e-b722fd089f11 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:43:09.056157 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-fa9210e4-0542-4fca-aa0e-b722fd089f11 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '39de6a6fff7c4cc08cc62aa86182d71d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1235d8b054b3405595fca761f08b73ae', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:43:09.067063 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-fa9210e4-0542-4fca-aa0e-b722fd089f11 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] 10.222.0.26 "GET /compute/v2.1/servers/5bdc36c3-1db5-4ff2-808d-1295f314d522" status: 200 len: 1319 microversion: 2.1 time: 0.420849 Oct 19 00:43:09.084252 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 213/328] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:43:08 2022] GET /compute/v2.1/servers/5bdc36c3-1db5-4ff2-808d-1295f314d522 => generated 1319 bytes in 438 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:43:09.262394 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-c25b35e7-e92d-48e3-b103-e445e79cb9a8 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:43:09.263926 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-c25b35e7-e92d-48e3-b103-e445e79cb9a8 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] [instance: f6051269-1d16-42db-8323-67a532dbcf40] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:43:09.277263 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-c25b35e7-e92d-48e3-b103-e445e79cb9a8 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:43:09.282031 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-c25b35e7-e92d-48e3-b103-e445e79cb9a8 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:43:09.283102 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-c25b35e7-e92d-48e3-b103-e445e79cb9a8 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:43:09.284745 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-c25b35e7-e92d-48e3-b103-e445e79cb9a8 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:43:09.285522 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-c25b35e7-e92d-48e3-b103-e445e79cb9a8 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:43:09.286254 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-c25b35e7-e92d-48e3-b103-e445e79cb9a8 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:43:09.602031 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-c25b35e7-e92d-48e3-b103-e445e79cb9a8 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd3f0d615187d4779bdd7da8c9f24c796', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0151d58abd047a692f5cc4e7d05bea5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:43:09.613044 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-c25b35e7-e92d-48e3-b103-e445e79cb9a8 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] 10.222.0.26 "GET /compute/v2.1/servers/f6051269-1d16-42db-8323-67a532dbcf40" status: 200 len: 1372 microversion: 2.1 time: 0.356085 Oct 19 00:43:09.627357 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 214/329] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:43:09 2022] GET /compute/v2.1/servers/f6051269-1d16-42db-8323-67a532dbcf40 => generated 1372 bytes in 371 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:43:09.644189 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-27facd8a-0f21-4fef-a3cc-a6d28361c1bb tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:43:09.646080 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-27facd8a-0f21-4fef-a3cc-a6d28361c1bb tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] [instance: b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:43:09.665090 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-27facd8a-0f21-4fef-a3cc-a6d28361c1bb tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:43:09.667570 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-27facd8a-0f21-4fef-a3cc-a6d28361c1bb tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:43:09.670423 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-27facd8a-0f21-4fef-a3cc-a6d28361c1bb tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:43:09.671744 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-27facd8a-0f21-4fef-a3cc-a6d28361c1bb tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:43:09.672455 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-27facd8a-0f21-4fef-a3cc-a6d28361c1bb tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:43:09.675134 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-27facd8a-0f21-4fef-a3cc-a6d28361c1bb tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:43:10.496657 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-27facd8a-0f21-4fef-a3cc-a6d28361c1bb tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:43:10.517083 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-27facd8a-0f21-4fef-a3cc-a6d28361c1bb tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:43:10.518837 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-27facd8a-0f21-4fef-a3cc-a6d28361c1bb tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:43:10.520351 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-27facd8a-0f21-4fef-a3cc-a6d28361c1bb tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] 10.222.0.26 "GET /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2" status: 200 len: 1739 microversion: 2.20 time: 0.890664 Oct 19 00:43:10.520977 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 215/330] 10.222.0.26 () {60 vars in 1277 bytes} [Wed Oct 19 00:43:09 2022] GET /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2 => generated 1739 bytes in 893 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 19 00:43:10.526427 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-aa8f7271-f0fd-47ca-a4e0-2875f84c14ff tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:43:10.527539 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-aa8f7271-f0fd-47ca-a4e0-2875f84c14ff tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] [instance: c5d02d64-d4e6-445d-a155-b875b8a8cce0] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:43:10.535762 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-aa8f7271-f0fd-47ca-a4e0-2875f84c14ff tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:43:10.536604 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-aa8f7271-f0fd-47ca-a4e0-2875f84c14ff tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:43:10.537404 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-aa8f7271-f0fd-47ca-a4e0-2875f84c14ff tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:43:10.538372 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-aa8f7271-f0fd-47ca-a4e0-2875f84c14ff tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:43:10.539077 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-aa8f7271-f0fd-47ca-a4e0-2875f84c14ff tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:43:10.539715 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-aa8f7271-f0fd-47ca-a4e0-2875f84c14ff tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:43:10.558038 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.wsgi [None req-aa8f7271-f0fd-47ca-a4e0-2875f84c14ff tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] HTTP exception thrown: Instance c5d02d64-d4e6-445d-a155-b875b8a8cce0 could not be found. Oct 19 00:43:10.560964 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-aa8f7271-f0fd-47ca-a4e0-2875f84c14ff tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Request method failure captured: Oct 19 00:43:10.560964 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: request: GET /compute/v2.1/servers/c5d02d64-d4e6-445d-a155-b875b8a8cce0 HTTP/1.1 Oct 19 00:43:10.560964 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Accept: application/json Oct 19 00:43:10.560964 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Accept-Encoding: identity Oct 19 00:43:10.560964 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Connection: close Oct 19 00:43:10.560964 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Content-Length: 0 Oct 19 00:43:10.560964 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Content-Type: application/json Oct 19 00:43:10.560964 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Host: 10.222.0.26 Oct 19 00:43:10.560964 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Openstack-System-Scope: None Oct 19 00:43:10.560964 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: User-Agent: python-urllib3/1.26.12 Oct 19 00:43:10.560964 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Auth-Token: gAAAAABjT0fWq8PufKfTNVGCB8UZN8rquspHB0M18f9BpiVGpFdRUCE9-sef668qbt4mHsvxBbayoK7Q4ESz7O8Fc51IujD1f1lyiIXi0P75C3OdhQ8yI0wWhqAReMDCiWL9XbVVxqVdF3iwbCfkbNEiXK-RJrPPtPv4_Nq7EBUKZW6gSE2wfW8 Oct 19 00:43:10.560964 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Domain-Id: None Oct 19 00:43:10.560964 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Domain-Name: None Oct 19 00:43:10.560964 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Identity-Status: Confirmed Oct 19 00:43:10.560964 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Is-Admin-Project: True Oct 19 00:43:10.560964 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Project-Domain-Id: default Oct 19 00:43:10.560964 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Project-Domain-Name: Default Oct 19 00:43:10.560964 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Project-Id: 173a52b79dab4962a4aa380c26f97d2d Oct 19 00:43:10.560964 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Project-Name: tempest-VolumesActionsTest-2037902882 Oct 19 00:43:10.560964 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Role: member,reader Oct 19 00:43:10.560964 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Roles: member,reader Oct 19 00:43:10.562368 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/173a52b79dab4962a4aa380c26f97d2d"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/173a52b79dab4962a4aa380c26f97d2d"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/173a52b79dab4962a4aa380c26f97d2d"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_173a52b79dab4962a4aa380c26f97d2d"}]}] Oct 19 00:43:10.562368 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Tenant: tempest-VolumesActionsTest-2037902882 Oct 19 00:43:10.562368 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Tenant-Id: 173a52b79dab4962a4aa380c26f97d2d Oct 19 00:43:10.562368 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Tenant-Name: tempest-VolumesActionsTest-2037902882 Oct 19 00:43:10.562368 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User: tempest-VolumesActionsTest-2037902882-project Oct 19 00:43:10.562368 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User-Domain-Id: default Oct 19 00:43:10.562368 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User-Domain-Name: Default Oct 19 00:43:10.562368 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User-Id: a08c9457231a447eb1c4130e68cfb31d Oct 19 00:43:10.562368 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User-Name: tempest-VolumesActionsTest-2037902882-project Oct 19 00:43:10.562368 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: method: > Oct 19 00:43:10.562368 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: exception: Instance c5d02d64-d4e6-445d-a155-b875b8a8cce0 could not be found. Oct 19 00:43:10.562368 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: action_args: {'id': 'c5d02d64-d4e6-445d-a155-b875b8a8cce0'} Oct 19 00:43:10.563541 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 19 00:43:10.563541 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 19 00:43:10.563541 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 19 00:43:10.563541 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 19 00:43:10.563541 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 19 00:43:10.563541 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 19 00:43:10.563541 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 19 00:43:10.563541 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 19 00:43:10.563541 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 19 00:43:10.563541 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 19 00:43:10.563541 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance c5d02d64-d4e6-445d-a155-b875b8a8cce0 could not be found. Oct 19 00:43:10.563541 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:43:10.563541 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 19 00:43:10.563541 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:43:10.563541 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 19 00:43:10.563541 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 19 00:43:10.563541 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 19 00:43:10.563541 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 19 00:43:10.563541 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 19 00:43:10.563541 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 19 00:43:10.564756 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 19 00:43:10.564756 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 19 00:43:10.564756 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 19 00:43:10.564756 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 19 00:43:10.564756 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 19 00:43:10.564756 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 19 00:43:10.564756 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 19 00:43:10.564756 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance c5d02d64-d4e6-445d-a155-b875b8a8cce0 could not be found. Oct 19 00:43:10.564756 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:43:10.564756 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 19 00:43:10.564756 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:43:10.564756 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 19 00:43:10.564756 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 19 00:43:10.564756 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 19 00:43:10.564756 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 19 00:43:10.564756 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 19 00:43:10.564756 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance c5d02d64-d4e6-445d-a155-b875b8a8cce0 could not be found. Oct 19 00:43:10.564756 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:43:10.567819 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-aa8f7271-f0fd-47ca-a4e0-2875f84c14ff tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] Returning 404 to user: Instance c5d02d64-d4e6-445d-a155-b875b8a8cce0 could not be found. {{(pid=125641) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 19 00:43:10.569064 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-aa8f7271-f0fd-47ca-a4e0-2875f84c14ff tempest-VolumesActionsTest-2037902882 tempest-VolumesActionsTest-2037902882-project] 10.222.0.26 "GET /compute/v2.1/servers/c5d02d64-d4e6-445d-a155-b875b8a8cce0" status: 404 len: 111 microversion: 2.1 time: 0.046333 Oct 19 00:43:10.570277 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 216/331] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:43:10 2022] GET /compute/v2.1/servers/c5d02d64-d4e6-445d-a155-b875b8a8cce0 => generated 111 bytes in 48 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 19 00:43:10.583417 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-8f9f2f39-be61-4879-a222-5c788b690f95 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:43:10.584748 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-8f9f2f39-be61-4879-a222-5c788b690f95 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] [instance: 5bdc36c3-1db5-4ff2-808d-1295f314d522] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:43:10.594740 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-8f9f2f39-be61-4879-a222-5c788b690f95 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:43:10.595987 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-8f9f2f39-be61-4879-a222-5c788b690f95 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:43:10.596675 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-8f9f2f39-be61-4879-a222-5c788b690f95 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:43:10.597622 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-8f9f2f39-be61-4879-a222-5c788b690f95 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:43:10.598294 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-8f9f2f39-be61-4879-a222-5c788b690f95 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:43:10.599627 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-8f9f2f39-be61-4879-a222-5c788b690f95 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:43:10.947952 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-8f9f2f39-be61-4879-a222-5c788b690f95 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '39de6a6fff7c4cc08cc62aa86182d71d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1235d8b054b3405595fca761f08b73ae', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:43:10.964881 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-8f9f2f39-be61-4879-a222-5c788b690f95 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] 10.222.0.26 "GET /compute/v2.1/servers/5bdc36c3-1db5-4ff2-808d-1295f314d522" status: 200 len: 1319 microversion: 2.1 time: 0.392883 Oct 19 00:43:10.965883 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 217/332] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:43:10 2022] GET /compute/v2.1/servers/5bdc36c3-1db5-4ff2-808d-1295f314d522 => generated 1319 bytes in 395 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:43:10.973190 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-8be476b8-09e9-4165-8b5b-052efdad8c16 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:43:10.989174 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-8be476b8-09e9-4165-8b5b-052efdad8c16 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] [instance: f6051269-1d16-42db-8323-67a532dbcf40] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:43:11.012278 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-8be476b8-09e9-4165-8b5b-052efdad8c16 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:43:11.013952 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-8be476b8-09e9-4165-8b5b-052efdad8c16 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:43:11.014391 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-8be476b8-09e9-4165-8b5b-052efdad8c16 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:43:11.015142 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-8be476b8-09e9-4165-8b5b-052efdad8c16 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:43:11.015620 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-8be476b8-09e9-4165-8b5b-052efdad8c16 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:43:11.016090 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-8be476b8-09e9-4165-8b5b-052efdad8c16 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:43:11.227345 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-8be476b8-09e9-4165-8b5b-052efdad8c16 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd3f0d615187d4779bdd7da8c9f24c796', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0151d58abd047a692f5cc4e7d05bea5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:43:11.237567 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-8be476b8-09e9-4165-8b5b-052efdad8c16 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] 10.222.0.26 "GET /compute/v2.1/servers/f6051269-1d16-42db-8323-67a532dbcf40" status: 200 len: 1372 microversion: 2.1 time: 0.268472 Oct 19 00:43:11.259449 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 218/333] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:43:10 2022] GET /compute/v2.1/servers/f6051269-1d16-42db-8323-67a532dbcf40 => generated 1372 bytes in 291 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:43:11.563917 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-c51e93db-bf62-4746-b02d-5f918da0bd5a tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:43:11.573052 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-c51e93db-bf62-4746-b02d-5f918da0bd5a tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] [instance: b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:43:11.623101 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-c51e93db-bf62-4746-b02d-5f918da0bd5a tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:43:11.632115 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-c51e93db-bf62-4746-b02d-5f918da0bd5a tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:43:11.634804 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-c51e93db-bf62-4746-b02d-5f918da0bd5a tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:43:11.636874 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-c51e93db-bf62-4746-b02d-5f918da0bd5a tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:43:11.637379 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-c51e93db-bf62-4746-b02d-5f918da0bd5a tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:43:11.643325 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-c51e93db-bf62-4746-b02d-5f918da0bd5a tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:43:11.973095 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-c51e93db-bf62-4746-b02d-5f918da0bd5a tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:43:12.007571 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-c51e93db-bf62-4746-b02d-5f918da0bd5a tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:43:12.016992 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-c51e93db-bf62-4746-b02d-5f918da0bd5a tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:43:12.019207 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-c51e93db-bf62-4746-b02d-5f918da0bd5a tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] 10.222.0.26 "GET /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2" status: 200 len: 1739 microversion: 2.20 time: 0.474742 Oct 19 00:43:12.019923 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 219/334] 10.222.0.26 () {60 vars in 1277 bytes} [Wed Oct 19 00:43:11 2022] GET /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2 => generated 1739 bytes in 477 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 19 00:43:12.033537 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-6bde92a8-bd7e-4aae-9510-bc0139e44730 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:43:12.035006 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-6bde92a8-bd7e-4aae-9510-bc0139e44730 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] [instance: 5bdc36c3-1db5-4ff2-808d-1295f314d522] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:43:12.058632 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-6bde92a8-bd7e-4aae-9510-bc0139e44730 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:43:12.061568 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-6bde92a8-bd7e-4aae-9510-bc0139e44730 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:43:12.063874 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-6bde92a8-bd7e-4aae-9510-bc0139e44730 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:43:12.078939 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-6bde92a8-bd7e-4aae-9510-bc0139e44730 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:43:12.084112 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-6bde92a8-bd7e-4aae-9510-bc0139e44730 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:43:12.086691 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-6bde92a8-bd7e-4aae-9510-bc0139e44730 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:43:12.439343 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-6bde92a8-bd7e-4aae-9510-bc0139e44730 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '39de6a6fff7c4cc08cc62aa86182d71d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1235d8b054b3405595fca761f08b73ae', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:43:12.450863 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-6bde92a8-bd7e-4aae-9510-bc0139e44730 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] 10.222.0.26 "GET /compute/v2.1/servers/5bdc36c3-1db5-4ff2-808d-1295f314d522" status: 200 len: 1319 microversion: 2.1 time: 0.428944 Oct 19 00:43:12.451453 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 220/335] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:43:12 2022] GET /compute/v2.1/servers/5bdc36c3-1db5-4ff2-808d-1295f314d522 => generated 1319 bytes in 431 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:43:12.457326 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-9cc15e7c-279e-4250-8517-8b1b9dd1ba46 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:43:12.458230 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-9cc15e7c-279e-4250-8517-8b1b9dd1ba46 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] [instance: f6051269-1d16-42db-8323-67a532dbcf40] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:43:12.465213 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-9cc15e7c-279e-4250-8517-8b1b9dd1ba46 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:43:12.466785 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-9cc15e7c-279e-4250-8517-8b1b9dd1ba46 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:43:12.467538 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-9cc15e7c-279e-4250-8517-8b1b9dd1ba46 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:43:12.468845 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-9cc15e7c-279e-4250-8517-8b1b9dd1ba46 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:43:12.469559 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-9cc15e7c-279e-4250-8517-8b1b9dd1ba46 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:43:12.470509 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-9cc15e7c-279e-4250-8517-8b1b9dd1ba46 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:43:12.693850 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-9cc15e7c-279e-4250-8517-8b1b9dd1ba46 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd3f0d615187d4779bdd7da8c9f24c796', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0151d58abd047a692f5cc4e7d05bea5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:43:12.704504 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-9cc15e7c-279e-4250-8517-8b1b9dd1ba46 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] 10.222.0.26 "GET /compute/v2.1/servers/f6051269-1d16-42db-8323-67a532dbcf40" status: 200 len: 1372 microversion: 2.1 time: 0.251540 Oct 19 00:43:12.705770 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 221/336] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:43:12 2022] GET /compute/v2.1/servers/f6051269-1d16-42db-8323-67a532dbcf40 => generated 1372 bytes in 253 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:43:12.722786 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [req-0a58d2f5-9e40-4f6b-aee5-b3a57074027c req-697dc963-d121-4a0c-8bc3-5e12494fa7f1 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "5bdc36c3-1db5-4ff2-808d-1295f314d522", "tag": "97ea2bef-8a45-4c17-8415-e62d303621cb"}]} {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 19 00:43:12.738961 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-0a58d2f5-9e40-4f6b-aee5-b3a57074027c req-697dc963-d121-4a0c-8bc3-5e12494fa7f1 service nova] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:43:12.739873 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-0a58d2f5-9e40-4f6b-aee5-b3a57074027c req-697dc963-d121-4a0c-8bc3-5e12494fa7f1 service nova] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:43:12.740373 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-0a58d2f5-9e40-4f6b-aee5-b3a57074027c req-697dc963-d121-4a0c-8bc3-5e12494fa7f1 service nova] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:43:12.753232 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.compute.server_external_events [req-0a58d2f5-9e40-4f6b-aee5-b3a57074027c req-697dc963-d121-4a0c-8bc3-5e12494fa7f1 service nova] Creating event network-changed:97ea2bef-8a45-4c17-8415-e62d303621cb for instance 5bdc36c3-1db5-4ff2-808d-1295f314d522 on hv-cinder-maste Oct 19 00:43:12.759097 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [req-0a58d2f5-9e40-4f6b-aee5-b3a57074027c req-697dc963-d121-4a0c-8bc3-5e12494fa7f1 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.051965 Oct 19 00:43:12.759961 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 222/337] 10.222.0.26 () {64 vars in 1317 bytes} [Wed Oct 19 00:43:12 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 19 00:43:13.054143 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-a94a9ee7-129d-40bc-a3f2-a52e23445f58 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:43:13.054880 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-a94a9ee7-129d-40bc-a3f2-a52e23445f58 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] [instance: b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:43:13.061429 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-a94a9ee7-129d-40bc-a3f2-a52e23445f58 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:43:13.061913 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-a94a9ee7-129d-40bc-a3f2-a52e23445f58 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:43:13.062449 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-a94a9ee7-129d-40bc-a3f2-a52e23445f58 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:43:13.063371 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-a94a9ee7-129d-40bc-a3f2-a52e23445f58 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:43:13.063835 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-a94a9ee7-129d-40bc-a3f2-a52e23445f58 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:43:13.064258 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-a94a9ee7-129d-40bc-a3f2-a52e23445f58 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:43:13.245978 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-a94a9ee7-129d-40bc-a3f2-a52e23445f58 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:43:13.260897 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-a94a9ee7-129d-40bc-a3f2-a52e23445f58 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:43:13.263513 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-a94a9ee7-129d-40bc-a3f2-a52e23445f58 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:43:13.265406 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-a94a9ee7-129d-40bc-a3f2-a52e23445f58 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] 10.222.0.26 "GET /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2" status: 200 len: 1739 microversion: 2.20 time: 0.215790 Oct 19 00:43:13.280544 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 223/338] 10.222.0.26 () {60 vars in 1277 bytes} [Wed Oct 19 00:43:13 2022] GET /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2 => generated 1739 bytes in 232 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 19 00:43:13.475094 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-2ad5a4ed-e123-4fbd-b6c4-1752b33cc596 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:43:13.475731 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-2ad5a4ed-e123-4fbd-b6c4-1752b33cc596 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] [instance: 5bdc36c3-1db5-4ff2-808d-1295f314d522] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:43:13.484218 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-2ad5a4ed-e123-4fbd-b6c4-1752b33cc596 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:43:13.484218 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-2ad5a4ed-e123-4fbd-b6c4-1752b33cc596 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:43:13.484702 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-2ad5a4ed-e123-4fbd-b6c4-1752b33cc596 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:43:13.485437 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-2ad5a4ed-e123-4fbd-b6c4-1752b33cc596 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:43:13.486268 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-2ad5a4ed-e123-4fbd-b6c4-1752b33cc596 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:43:13.486405 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-2ad5a4ed-e123-4fbd-b6c4-1752b33cc596 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:43:13.659075 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-2ad5a4ed-e123-4fbd-b6c4-1752b33cc596 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '39de6a6fff7c4cc08cc62aa86182d71d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1235d8b054b3405595fca761f08b73ae', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:43:13.672006 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-2ad5a4ed-e123-4fbd-b6c4-1752b33cc596 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] 10.222.0.26 "GET /compute/v2.1/servers/5bdc36c3-1db5-4ff2-808d-1295f314d522" status: 200 len: 1319 microversion: 2.1 time: 0.201765 Oct 19 00:43:13.673557 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 224/339] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:43:13 2022] GET /compute/v2.1/servers/5bdc36c3-1db5-4ff2-808d-1295f314d522 => generated 1319 bytes in 204 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:43:13.728434 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-e794e6a0-01cf-4899-ae50-987c5ecf8d87 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:43:13.730136 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-e794e6a0-01cf-4899-ae50-987c5ecf8d87 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] [instance: f6051269-1d16-42db-8323-67a532dbcf40] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:43:13.740443 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-e794e6a0-01cf-4899-ae50-987c5ecf8d87 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:43:13.741044 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-e794e6a0-01cf-4899-ae50-987c5ecf8d87 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:43:13.741569 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-e794e6a0-01cf-4899-ae50-987c5ecf8d87 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:43:13.742574 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-e794e6a0-01cf-4899-ae50-987c5ecf8d87 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:43:13.743429 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-e794e6a0-01cf-4899-ae50-987c5ecf8d87 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:43:13.744712 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-e794e6a0-01cf-4899-ae50-987c5ecf8d87 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:43:13.931247 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-e794e6a0-01cf-4899-ae50-987c5ecf8d87 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd3f0d615187d4779bdd7da8c9f24c796', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0151d58abd047a692f5cc4e7d05bea5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:43:13.942761 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-e794e6a0-01cf-4899-ae50-987c5ecf8d87 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] 10.222.0.26 "GET /compute/v2.1/servers/f6051269-1d16-42db-8323-67a532dbcf40" status: 200 len: 1372 microversion: 2.1 time: 0.218001 Oct 19 00:43:13.958924 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 225/340] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:43:13 2022] GET /compute/v2.1/servers/f6051269-1d16-42db-8323-67a532dbcf40 => generated 1372 bytes in 235 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:43:14.293620 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-53fc49d3-a3f4-4416-90c1-21efe105f26d tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:43:14.294181 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-53fc49d3-a3f4-4416-90c1-21efe105f26d tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] [instance: b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:43:14.301395 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-53fc49d3-a3f4-4416-90c1-21efe105f26d tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:43:14.302294 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-53fc49d3-a3f4-4416-90c1-21efe105f26d tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:43:14.302761 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-53fc49d3-a3f4-4416-90c1-21efe105f26d tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:43:14.303460 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-53fc49d3-a3f4-4416-90c1-21efe105f26d tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:43:14.303911 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-53fc49d3-a3f4-4416-90c1-21efe105f26d tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:43:14.304329 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-53fc49d3-a3f4-4416-90c1-21efe105f26d tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:43:14.505288 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-53fc49d3-a3f4-4416-90c1-21efe105f26d tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:43:14.517140 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-53fc49d3-a3f4-4416-90c1-21efe105f26d tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:43:14.519000 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-53fc49d3-a3f4-4416-90c1-21efe105f26d tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:43:14.520621 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-53fc49d3-a3f4-4416-90c1-21efe105f26d tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] 10.222.0.26 "GET /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2" status: 200 len: 1739 microversion: 2.20 time: 0.231529 Oct 19 00:43:14.521748 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 226/341] 10.222.0.26 () {60 vars in 1277 bytes} [Wed Oct 19 00:43:14 2022] GET /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2 => generated 1739 bytes in 233 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 19 00:43:14.699676 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-73e4e40f-b8c9-4181-8aec-6501083695b0 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:43:14.700815 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-73e4e40f-b8c9-4181-8aec-6501083695b0 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] [instance: 5bdc36c3-1db5-4ff2-808d-1295f314d522] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:43:14.711803 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-73e4e40f-b8c9-4181-8aec-6501083695b0 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:43:14.713109 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-73e4e40f-b8c9-4181-8aec-6501083695b0 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:43:14.713794 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-73e4e40f-b8c9-4181-8aec-6501083695b0 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:43:14.714833 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-73e4e40f-b8c9-4181-8aec-6501083695b0 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:43:14.715566 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-73e4e40f-b8c9-4181-8aec-6501083695b0 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:43:14.716222 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-73e4e40f-b8c9-4181-8aec-6501083695b0 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:43:14.950189 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-73e4e40f-b8c9-4181-8aec-6501083695b0 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '39de6a6fff7c4cc08cc62aa86182d71d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1235d8b054b3405595fca761f08b73ae', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:43:14.961885 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-73e4e40f-b8c9-4181-8aec-6501083695b0 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] 10.222.0.26 "GET /compute/v2.1/servers/5bdc36c3-1db5-4ff2-808d-1295f314d522" status: 200 len: 1319 microversion: 2.1 time: 0.266576 Oct 19 00:43:14.975653 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 227/342] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:43:14 2022] GET /compute/v2.1/servers/5bdc36c3-1db5-4ff2-808d-1295f314d522 => generated 1319 bytes in 281 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:43:14.985775 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-925a038f-b798-49da-8028-5564c78daf08 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:43:14.986406 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-925a038f-b798-49da-8028-5564c78daf08 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] [instance: f6051269-1d16-42db-8323-67a532dbcf40] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:43:14.992900 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-925a038f-b798-49da-8028-5564c78daf08 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:43:14.993682 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-925a038f-b798-49da-8028-5564c78daf08 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:43:14.994304 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-925a038f-b798-49da-8028-5564c78daf08 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:43:14.996062 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-925a038f-b798-49da-8028-5564c78daf08 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:43:14.996789 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-925a038f-b798-49da-8028-5564c78daf08 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:43:14.998096 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-925a038f-b798-49da-8028-5564c78daf08 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:43:15.022323 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG dbcounter [-] [125641] Writing DB stats nova_cell1:SELECT=100 {{(pid=125641) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 00:43:15.254541 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-925a038f-b798-49da-8028-5564c78daf08 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd3f0d615187d4779bdd7da8c9f24c796', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0151d58abd047a692f5cc4e7d05bea5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:43:15.265541 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-925a038f-b798-49da-8028-5564c78daf08 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] 10.222.0.26 "GET /compute/v2.1/servers/f6051269-1d16-42db-8323-67a532dbcf40" status: 200 len: 1372 microversion: 2.1 time: 0.288641 Oct 19 00:43:15.266266 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 228/343] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:43:14 2022] GET /compute/v2.1/servers/f6051269-1d16-42db-8323-67a532dbcf40 => generated 1372 bytes in 290 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:43:15.561733 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-73857f2f-53d6-4d04-9029-d9cffef52c85 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:43:15.562737 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-73857f2f-53d6-4d04-9029-d9cffef52c85 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] [instance: b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:43:15.572929 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-73857f2f-53d6-4d04-9029-d9cffef52c85 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:43:15.573698 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-73857f2f-53d6-4d04-9029-d9cffef52c85 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:43:15.574134 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-73857f2f-53d6-4d04-9029-d9cffef52c85 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:43:15.586455 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-73857f2f-53d6-4d04-9029-d9cffef52c85 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:43:15.587247 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-73857f2f-53d6-4d04-9029-d9cffef52c85 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:43:15.588113 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-73857f2f-53d6-4d04-9029-d9cffef52c85 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:43:15.766583 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-73857f2f-53d6-4d04-9029-d9cffef52c85 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:43:15.783979 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-73857f2f-53d6-4d04-9029-d9cffef52c85 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:43:15.785792 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-73857f2f-53d6-4d04-9029-d9cffef52c85 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:43:15.787370 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-73857f2f-53d6-4d04-9029-d9cffef52c85 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] 10.222.0.26 "GET /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2" status: 200 len: 1739 microversion: 2.20 time: 0.229818 Oct 19 00:43:15.788402 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 229/344] 10.222.0.26 () {60 vars in 1277 bytes} [Wed Oct 19 00:43:15 2022] GET /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2 => generated 1739 bytes in 237 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 19 00:43:15.991533 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-66fd1fa5-7a47-4fe1-8d14-a3a08a06b5d1 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:43:15.992259 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-66fd1fa5-7a47-4fe1-8d14-a3a08a06b5d1 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] [instance: 5bdc36c3-1db5-4ff2-808d-1295f314d522] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:43:16.006517 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-66fd1fa5-7a47-4fe1-8d14-a3a08a06b5d1 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:43:16.007021 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-66fd1fa5-7a47-4fe1-8d14-a3a08a06b5d1 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:43:16.007457 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-66fd1fa5-7a47-4fe1-8d14-a3a08a06b5d1 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:43:16.009007 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-66fd1fa5-7a47-4fe1-8d14-a3a08a06b5d1 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:43:16.009472 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-66fd1fa5-7a47-4fe1-8d14-a3a08a06b5d1 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:43:16.009903 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-66fd1fa5-7a47-4fe1-8d14-a3a08a06b5d1 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:43:16.210588 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-66fd1fa5-7a47-4fe1-8d14-a3a08a06b5d1 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '39de6a6fff7c4cc08cc62aa86182d71d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1235d8b054b3405595fca761f08b73ae', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:43:16.219267 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-66fd1fa5-7a47-4fe1-8d14-a3a08a06b5d1 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] 10.222.0.26 "GET /compute/v2.1/servers/5bdc36c3-1db5-4ff2-808d-1295f314d522" status: 200 len: 1319 microversion: 2.1 time: 0.232298 Oct 19 00:43:16.220672 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 230/345] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:43:15 2022] GET /compute/v2.1/servers/5bdc36c3-1db5-4ff2-808d-1295f314d522 => generated 1319 bytes in 234 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 19 00:43:16.292685 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-2d9264c3-6a95-409d-91d0-567df8139536 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:43:16.293697 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-2d9264c3-6a95-409d-91d0-567df8139536 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] [instance: f6051269-1d16-42db-8323-67a532dbcf40] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:43:16.303859 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-2d9264c3-6a95-409d-91d0-567df8139536 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:43:16.304685 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-2d9264c3-6a95-409d-91d0-567df8139536 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:43:16.305445 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-2d9264c3-6a95-409d-91d0-567df8139536 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:43:16.306246 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-2d9264c3-6a95-409d-91d0-567df8139536 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:43:16.306967 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-2d9264c3-6a95-409d-91d0-567df8139536 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:43:16.307703 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-2d9264c3-6a95-409d-91d0-567df8139536 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:43:16.468770 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-2d9264c3-6a95-409d-91d0-567df8139536 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd3f0d615187d4779bdd7da8c9f24c796', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0151d58abd047a692f5cc4e7d05bea5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:43:16.478594 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-2d9264c3-6a95-409d-91d0-567df8139536 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] 10.222.0.26 "GET /compute/v2.1/servers/f6051269-1d16-42db-8323-67a532dbcf40" status: 200 len: 1372 microversion: 2.1 time: 0.191072 Oct 19 00:43:16.479075 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 231/346] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:43:16 2022] GET /compute/v2.1/servers/f6051269-1d16-42db-8323-67a532dbcf40 => generated 1372 bytes in 193 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:43:16.817884 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-6893c5b1-1376-4f55-8ae1-e2912601b0c6 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:43:16.818731 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-6893c5b1-1376-4f55-8ae1-e2912601b0c6 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] [instance: b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:43:16.830583 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-6893c5b1-1376-4f55-8ae1-e2912601b0c6 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:43:16.831429 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-6893c5b1-1376-4f55-8ae1-e2912601b0c6 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:43:16.832302 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-6893c5b1-1376-4f55-8ae1-e2912601b0c6 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:43:16.833231 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-6893c5b1-1376-4f55-8ae1-e2912601b0c6 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:43:16.833698 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-6893c5b1-1376-4f55-8ae1-e2912601b0c6 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:43:16.835664 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-6893c5b1-1376-4f55-8ae1-e2912601b0c6 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:43:17.076617 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-6893c5b1-1376-4f55-8ae1-e2912601b0c6 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:43:17.086926 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-6893c5b1-1376-4f55-8ae1-e2912601b0c6 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:43:17.089206 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-6893c5b1-1376-4f55-8ae1-e2912601b0c6 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:43:17.091539 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-6893c5b1-1376-4f55-8ae1-e2912601b0c6 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] 10.222.0.26 "GET /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2" status: 200 len: 1739 microversion: 2.20 time: 0.277508 Oct 19 00:43:17.092726 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 232/347] 10.222.0.26 () {60 vars in 1277 bytes} [Wed Oct 19 00:43:16 2022] GET /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2 => generated 1739 bytes in 280 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 19 00:43:17.244495 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-26b07dfc-e108-4a86-afd1-83304c6d5af2 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:43:17.245390 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-26b07dfc-e108-4a86-afd1-83304c6d5af2 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] [instance: 5bdc36c3-1db5-4ff2-808d-1295f314d522] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:43:17.252292 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-26b07dfc-e108-4a86-afd1-83304c6d5af2 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:43:17.253076 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-26b07dfc-e108-4a86-afd1-83304c6d5af2 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:43:17.253700 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-26b07dfc-e108-4a86-afd1-83304c6d5af2 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:43:17.254680 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-26b07dfc-e108-4a86-afd1-83304c6d5af2 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:43:17.255353 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-26b07dfc-e108-4a86-afd1-83304c6d5af2 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:43:17.256022 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-26b07dfc-e108-4a86-afd1-83304c6d5af2 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:43:17.522388 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-26b07dfc-e108-4a86-afd1-83304c6d5af2 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '39de6a6fff7c4cc08cc62aa86182d71d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1235d8b054b3405595fca761f08b73ae', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:43:17.533699 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-26b07dfc-e108-4a86-afd1-83304c6d5af2 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] 10.222.0.26 "GET /compute/v2.1/servers/5bdc36c3-1db5-4ff2-808d-1295f314d522" status: 200 len: 1319 microversion: 2.1 time: 0.293358 Oct 19 00:43:17.546362 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 233/348] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:43:17 2022] GET /compute/v2.1/servers/5bdc36c3-1db5-4ff2-808d-1295f314d522 => generated 1319 bytes in 306 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:43:17.564776 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-be48c6be-1262-4bed-823b-46485013991e tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:43:17.565344 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-be48c6be-1262-4bed-823b-46485013991e tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] [instance: f6051269-1d16-42db-8323-67a532dbcf40] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:43:17.582168 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-be48c6be-1262-4bed-823b-46485013991e tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:43:17.582748 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-be48c6be-1262-4bed-823b-46485013991e tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:43:17.583188 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-be48c6be-1262-4bed-823b-46485013991e tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:43:17.583906 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-be48c6be-1262-4bed-823b-46485013991e tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:43:17.584365 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-be48c6be-1262-4bed-823b-46485013991e tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:43:17.584791 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-be48c6be-1262-4bed-823b-46485013991e tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:43:17.817668 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-be48c6be-1262-4bed-823b-46485013991e tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd3f0d615187d4779bdd7da8c9f24c796', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0151d58abd047a692f5cc4e7d05bea5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:43:17.828333 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-be48c6be-1262-4bed-823b-46485013991e tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] 10.222.0.26 "GET /compute/v2.1/servers/f6051269-1d16-42db-8323-67a532dbcf40" status: 200 len: 1372 microversion: 2.1 time: 0.280621 Oct 19 00:43:17.829067 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 234/349] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:43:17 2022] GET /compute/v2.1/servers/f6051269-1d16-42db-8323-67a532dbcf40 => generated 1372 bytes in 282 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:43:17.842855 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [req-1611f022-7302-4bbf-94e7-63afa9895343 req-2a2ca9cb-ec2b-4bfc-a616-1cb7f17a0a47 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "f6051269-1d16-42db-8323-67a532dbcf40", "name": "network-vif-plugged", "status": "completed", "tag": "667c66be-7422-49b6-8e96-6c36f067b697"}]} {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 19 00:43:17.847247 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-1611f022-7302-4bbf-94e7-63afa9895343 req-2a2ca9cb-ec2b-4bfc-a616-1cb7f17a0a47 service nova] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:43:17.848004 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-1611f022-7302-4bbf-94e7-63afa9895343 req-2a2ca9cb-ec2b-4bfc-a616-1cb7f17a0a47 service nova] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:43:17.848793 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-1611f022-7302-4bbf-94e7-63afa9895343 req-2a2ca9cb-ec2b-4bfc-a616-1cb7f17a0a47 service nova] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:43:17.870509 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.compute.server_external_events [req-1611f022-7302-4bbf-94e7-63afa9895343 req-2a2ca9cb-ec2b-4bfc-a616-1cb7f17a0a47 service nova] Creating event network-vif-plugged:667c66be-7422-49b6-8e96-6c36f067b697 for instance f6051269-1d16-42db-8323-67a532dbcf40 on hv-cinder-maste Oct 19 00:43:17.876966 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [req-1611f022-7302-4bbf-94e7-63afa9895343 req-2a2ca9cb-ec2b-4bfc-a616-1cb7f17a0a47 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.046592 Oct 19 00:43:17.878240 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 235/350] 10.222.0.26 () {64 vars in 1317 bytes} [Wed Oct 19 00:43:17 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 19 00:43:18.120143 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-8ad668c8-1eb1-425d-b505-e8bdba589eb8 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:43:18.120934 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-8ad668c8-1eb1-425d-b505-e8bdba589eb8 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] [instance: b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:43:18.132099 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-8ad668c8-1eb1-425d-b505-e8bdba589eb8 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:43:18.132921 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-8ad668c8-1eb1-425d-b505-e8bdba589eb8 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:43:18.133639 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-8ad668c8-1eb1-425d-b505-e8bdba589eb8 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:43:18.134549 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-8ad668c8-1eb1-425d-b505-e8bdba589eb8 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:43:18.135220 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-8ad668c8-1eb1-425d-b505-e8bdba589eb8 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:43:18.135899 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-8ad668c8-1eb1-425d-b505-e8bdba589eb8 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:43:18.306604 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-8ad668c8-1eb1-425d-b505-e8bdba589eb8 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:43:18.317689 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-8ad668c8-1eb1-425d-b505-e8bdba589eb8 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:43:18.319520 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-8ad668c8-1eb1-425d-b505-e8bdba589eb8 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:43:18.321040 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-8ad668c8-1eb1-425d-b505-e8bdba589eb8 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] 10.222.0.26 "GET /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2" status: 200 len: 1721 microversion: 2.20 time: 0.206276 Oct 19 00:43:18.335827 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 236/351] 10.222.0.26 () {60 vars in 1277 bytes} [Wed Oct 19 00:43:18 2022] GET /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2 => generated 1721 bytes in 222 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 19 00:43:18.567552 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-2f45185a-28e6-4cde-9be4-c8bb96e48271 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:43:18.568537 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-2f45185a-28e6-4cde-9be4-c8bb96e48271 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] [instance: 5bdc36c3-1db5-4ff2-808d-1295f314d522] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:43:18.580840 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-2f45185a-28e6-4cde-9be4-c8bb96e48271 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:43:18.581327 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-2f45185a-28e6-4cde-9be4-c8bb96e48271 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:43:18.581946 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-2f45185a-28e6-4cde-9be4-c8bb96e48271 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:43:18.582854 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-2f45185a-28e6-4cde-9be4-c8bb96e48271 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:43:18.583458 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-2f45185a-28e6-4cde-9be4-c8bb96e48271 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:43:18.584056 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-2f45185a-28e6-4cde-9be4-c8bb96e48271 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:43:18.760180 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-2f45185a-28e6-4cde-9be4-c8bb96e48271 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '39de6a6fff7c4cc08cc62aa86182d71d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1235d8b054b3405595fca761f08b73ae', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:43:18.770878 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-2f45185a-28e6-4cde-9be4-c8bb96e48271 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] 10.222.0.26 "GET /compute/v2.1/servers/5bdc36c3-1db5-4ff2-808d-1295f314d522" status: 200 len: 1319 microversion: 2.1 time: 0.216378 Oct 19 00:43:18.771597 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 237/352] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:43:18 2022] GET /compute/v2.1/servers/5bdc36c3-1db5-4ff2-808d-1295f314d522 => generated 1319 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:43:18.860364 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-4c2efc09-da62-40e2-8a90-8671463a0889 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:43:18.861479 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-4c2efc09-da62-40e2-8a90-8671463a0889 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] [instance: f6051269-1d16-42db-8323-67a532dbcf40] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:43:18.869254 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-4c2efc09-da62-40e2-8a90-8671463a0889 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:43:18.870210 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-4c2efc09-da62-40e2-8a90-8671463a0889 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:43:18.871051 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-4c2efc09-da62-40e2-8a90-8671463a0889 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:43:18.872072 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-4c2efc09-da62-40e2-8a90-8671463a0889 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:43:18.873000 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-4c2efc09-da62-40e2-8a90-8671463a0889 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:43:18.873675 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-4c2efc09-da62-40e2-8a90-8671463a0889 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:43:19.065242 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-4c2efc09-da62-40e2-8a90-8671463a0889 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd3f0d615187d4779bdd7da8c9f24c796', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0151d58abd047a692f5cc4e7d05bea5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:43:19.077432 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-4c2efc09-da62-40e2-8a90-8671463a0889 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] 10.222.0.26 "GET /compute/v2.1/servers/f6051269-1d16-42db-8323-67a532dbcf40" status: 200 len: 1372 microversion: 2.1 time: 0.221864 Oct 19 00:43:19.078367 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 238/353] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:43:18 2022] GET /compute/v2.1/servers/f6051269-1d16-42db-8323-67a532dbcf40 => generated 1372 bytes in 224 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 19 00:43:19.357536 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-a7dd195a-02f0-4b2a-89fc-c6816c518b3a tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:43:19.360256 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-a7dd195a-02f0-4b2a-89fc-c6816c518b3a tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] [instance: b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:43:19.377243 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-a7dd195a-02f0-4b2a-89fc-c6816c518b3a tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:43:19.379099 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-a7dd195a-02f0-4b2a-89fc-c6816c518b3a tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:43:19.379776 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-a7dd195a-02f0-4b2a-89fc-c6816c518b3a tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:43:19.380687 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-a7dd195a-02f0-4b2a-89fc-c6816c518b3a tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:43:19.381330 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-a7dd195a-02f0-4b2a-89fc-c6816c518b3a tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:43:19.382524 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-a7dd195a-02f0-4b2a-89fc-c6816c518b3a tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:43:19.683529 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-a7dd195a-02f0-4b2a-89fc-c6816c518b3a tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:43:19.722646 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-a7dd195a-02f0-4b2a-89fc-c6816c518b3a tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:43:19.725400 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-a7dd195a-02f0-4b2a-89fc-c6816c518b3a tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:43:19.727344 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-a7dd195a-02f0-4b2a-89fc-c6816c518b3a tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] 10.222.0.26 "GET /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2" status: 200 len: 1721 microversion: 2.20 time: 0.379531 Oct 19 00:43:19.727930 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 239/354] 10.222.0.26 () {60 vars in 1277 bytes} [Wed Oct 19 00:43:19 2022] GET /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2 => generated 1721 bytes in 381 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 19 00:43:19.797591 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-defd6831-bdf9-49e5-9a59-aa1b76afdd5e tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:43:19.798221 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-defd6831-bdf9-49e5-9a59-aa1b76afdd5e tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] [instance: 5bdc36c3-1db5-4ff2-808d-1295f314d522] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:43:19.804866 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-defd6831-bdf9-49e5-9a59-aa1b76afdd5e tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:43:19.805363 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-defd6831-bdf9-49e5-9a59-aa1b76afdd5e tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:43:19.805804 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-defd6831-bdf9-49e5-9a59-aa1b76afdd5e tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:43:19.806702 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-defd6831-bdf9-49e5-9a59-aa1b76afdd5e tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:43:19.807180 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-defd6831-bdf9-49e5-9a59-aa1b76afdd5e tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:43:19.807592 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-defd6831-bdf9-49e5-9a59-aa1b76afdd5e tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:43:19.967407 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-defd6831-bdf9-49e5-9a59-aa1b76afdd5e tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '39de6a6fff7c4cc08cc62aa86182d71d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1235d8b054b3405595fca761f08b73ae', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:43:19.976458 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-defd6831-bdf9-49e5-9a59-aa1b76afdd5e tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] 10.222.0.26 "GET /compute/v2.1/servers/5bdc36c3-1db5-4ff2-808d-1295f314d522" status: 200 len: 1319 microversion: 2.1 time: 0.184327 Oct 19 00:43:19.977382 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 240/355] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:43:19 2022] GET /compute/v2.1/servers/5bdc36c3-1db5-4ff2-808d-1295f314d522 => generated 1319 bytes in 186 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:43:20.104457 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-cb0039cc-36a7-49a0-81c2-ffe7f6bbddf4 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:43:20.105285 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-cb0039cc-36a7-49a0-81c2-ffe7f6bbddf4 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] [instance: f6051269-1d16-42db-8323-67a532dbcf40] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:43:20.125099 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-cb0039cc-36a7-49a0-81c2-ffe7f6bbddf4 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:43:20.125567 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-cb0039cc-36a7-49a0-81c2-ffe7f6bbddf4 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:43:20.127214 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-cb0039cc-36a7-49a0-81c2-ffe7f6bbddf4 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:43:20.131742 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-cb0039cc-36a7-49a0-81c2-ffe7f6bbddf4 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:43:20.133623 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-cb0039cc-36a7-49a0-81c2-ffe7f6bbddf4 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:43:20.134161 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-cb0039cc-36a7-49a0-81c2-ffe7f6bbddf4 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:43:20.307125 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-cb0039cc-36a7-49a0-81c2-ffe7f6bbddf4 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd3f0d615187d4779bdd7da8c9f24c796', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0151d58abd047a692f5cc4e7d05bea5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:43:20.317126 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-cb0039cc-36a7-49a0-81c2-ffe7f6bbddf4 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] 10.222.0.26 "GET /compute/v2.1/servers/f6051269-1d16-42db-8323-67a532dbcf40" status: 200 len: 1372 microversion: 2.1 time: 0.216839 Oct 19 00:43:20.317762 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 241/356] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:43:20 2022] GET /compute/v2.1/servers/f6051269-1d16-42db-8323-67a532dbcf40 => generated 1372 bytes in 219 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 19 00:43:20.754168 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-5685d8d2-80dc-42bb-bf9c-c31fed5c87d3 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:43:20.755754 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-5685d8d2-80dc-42bb-bf9c-c31fed5c87d3 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] [instance: b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:43:20.762162 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-5685d8d2-80dc-42bb-bf9c-c31fed5c87d3 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:43:20.763098 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-5685d8d2-80dc-42bb-bf9c-c31fed5c87d3 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:43:20.763636 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-5685d8d2-80dc-42bb-bf9c-c31fed5c87d3 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:43:20.764486 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-5685d8d2-80dc-42bb-bf9c-c31fed5c87d3 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:43:20.765071 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-5685d8d2-80dc-42bb-bf9c-c31fed5c87d3 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:43:20.765518 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-5685d8d2-80dc-42bb-bf9c-c31fed5c87d3 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:43:20.942692 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-5685d8d2-80dc-42bb-bf9c-c31fed5c87d3 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:43:20.952014 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-5685d8d2-80dc-42bb-bf9c-c31fed5c87d3 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:43:20.953675 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-5685d8d2-80dc-42bb-bf9c-c31fed5c87d3 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:43:20.955367 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-5685d8d2-80dc-42bb-bf9c-c31fed5c87d3 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] 10.222.0.26 "GET /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2" status: 200 len: 1721 microversion: 2.20 time: 0.205379 Oct 19 00:43:20.970975 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 242/357] 10.222.0.26 () {60 vars in 1277 bytes} [Wed Oct 19 00:43:20 2022] GET /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2 => generated 1721 bytes in 221 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 19 00:43:21.009812 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-cc26db8d-922c-4cbb-a6d9-71e9170289b1 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:43:21.010752 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-cc26db8d-922c-4cbb-a6d9-71e9170289b1 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] [instance: 5bdc36c3-1db5-4ff2-808d-1295f314d522] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:43:21.017241 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-cc26db8d-922c-4cbb-a6d9-71e9170289b1 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:43:21.017646 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-cc26db8d-922c-4cbb-a6d9-71e9170289b1 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:43:21.018269 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-cc26db8d-922c-4cbb-a6d9-71e9170289b1 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:43:21.018995 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-cc26db8d-922c-4cbb-a6d9-71e9170289b1 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:43:21.019451 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-cc26db8d-922c-4cbb-a6d9-71e9170289b1 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:43:21.019867 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-cc26db8d-922c-4cbb-a6d9-71e9170289b1 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:43:21.189315 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-cc26db8d-922c-4cbb-a6d9-71e9170289b1 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '39de6a6fff7c4cc08cc62aa86182d71d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1235d8b054b3405595fca761f08b73ae', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:43:21.198748 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-cc26db8d-922c-4cbb-a6d9-71e9170289b1 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] 10.222.0.26 "GET /compute/v2.1/servers/5bdc36c3-1db5-4ff2-808d-1295f314d522" status: 200 len: 1319 microversion: 2.1 time: 0.193121 Oct 19 00:43:21.199759 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 243/358] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:43:21 2022] GET /compute/v2.1/servers/5bdc36c3-1db5-4ff2-808d-1295f314d522 => generated 1319 bytes in 195 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 19 00:43:21.341633 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-5854c64d-07cc-4f10-b625-6f29ce5007f8 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:43:21.342278 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-5854c64d-07cc-4f10-b625-6f29ce5007f8 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] [instance: f6051269-1d16-42db-8323-67a532dbcf40] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:43:21.351554 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-5854c64d-07cc-4f10-b625-6f29ce5007f8 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:43:21.352059 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-5854c64d-07cc-4f10-b625-6f29ce5007f8 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:43:21.352501 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-5854c64d-07cc-4f10-b625-6f29ce5007f8 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:43:21.353227 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-5854c64d-07cc-4f10-b625-6f29ce5007f8 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:43:21.353690 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-5854c64d-07cc-4f10-b625-6f29ce5007f8 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:43:21.354117 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-5854c64d-07cc-4f10-b625-6f29ce5007f8 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:43:21.540205 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-5854c64d-07cc-4f10-b625-6f29ce5007f8 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd3f0d615187d4779bdd7da8c9f24c796', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0151d58abd047a692f5cc4e7d05bea5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:43:21.549115 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-5854c64d-07cc-4f10-b625-6f29ce5007f8 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] 10.222.0.26 "GET /compute/v2.1/servers/f6051269-1d16-42db-8323-67a532dbcf40" status: 200 len: 1372 microversion: 2.1 time: 0.211515 Oct 19 00:43:21.549680 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 244/359] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:43:21 2022] GET /compute/v2.1/servers/f6051269-1d16-42db-8323-67a532dbcf40 => generated 1372 bytes in 213 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 19 00:43:21.985102 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-e84b783e-05b9-4d95-81b8-1b51ae214c70 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:43:21.986690 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-e84b783e-05b9-4d95-81b8-1b51ae214c70 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] [instance: b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:43:21.993797 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-e84b783e-05b9-4d95-81b8-1b51ae214c70 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:43:21.994741 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-e84b783e-05b9-4d95-81b8-1b51ae214c70 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:43:21.995446 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-e84b783e-05b9-4d95-81b8-1b51ae214c70 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:43:21.996499 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-e84b783e-05b9-4d95-81b8-1b51ae214c70 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:43:21.997450 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-e84b783e-05b9-4d95-81b8-1b51ae214c70 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:43:21.998645 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-e84b783e-05b9-4d95-81b8-1b51ae214c70 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:43:22.161525 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-e84b783e-05b9-4d95-81b8-1b51ae214c70 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:43:22.166196 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG dbcounter [-] [125641] Writing DB stats nova_cell1:SELECT=100 {{(pid=125641) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 00:43:22.178219 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-e84b783e-05b9-4d95-81b8-1b51ae214c70 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:43:22.179700 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-e84b783e-05b9-4d95-81b8-1b51ae214c70 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:43:22.180885 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-e84b783e-05b9-4d95-81b8-1b51ae214c70 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] 10.222.0.26 "GET /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2" status: 200 len: 1721 microversion: 2.20 time: 0.201057 Oct 19 00:43:22.181831 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 245/360] 10.222.0.26 () {60 vars in 1277 bytes} [Wed Oct 19 00:43:21 2022] GET /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2 => generated 1721 bytes in 203 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 19 00:43:22.228312 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-d39426f5-c3e6-4349-8f2e-5f138fb4b9bd tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:43:22.230338 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-d39426f5-c3e6-4349-8f2e-5f138fb4b9bd tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] [instance: 5bdc36c3-1db5-4ff2-808d-1295f314d522] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:43:22.237369 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d39426f5-c3e6-4349-8f2e-5f138fb4b9bd tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:43:22.237895 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d39426f5-c3e6-4349-8f2e-5f138fb4b9bd tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:43:22.238432 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d39426f5-c3e6-4349-8f2e-5f138fb4b9bd tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:43:22.239879 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d39426f5-c3e6-4349-8f2e-5f138fb4b9bd tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:43:22.240500 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d39426f5-c3e6-4349-8f2e-5f138fb4b9bd tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:43:22.241082 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d39426f5-c3e6-4349-8f2e-5f138fb4b9bd tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:43:22.422562 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-d39426f5-c3e6-4349-8f2e-5f138fb4b9bd tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '39de6a6fff7c4cc08cc62aa86182d71d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1235d8b054b3405595fca761f08b73ae', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:43:22.432735 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-d39426f5-c3e6-4349-8f2e-5f138fb4b9bd tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] 10.222.0.26 "GET /compute/v2.1/servers/5bdc36c3-1db5-4ff2-808d-1295f314d522" status: 200 len: 1319 microversion: 2.1 time: 0.209274 Oct 19 00:43:22.433510 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 246/361] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:43:22 2022] GET /compute/v2.1/servers/5bdc36c3-1db5-4ff2-808d-1295f314d522 => generated 1319 bytes in 211 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 19 00:43:22.573234 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-3925dbd6-88f0-44bb-94ac-8204b77644f7 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:43:22.573925 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-3925dbd6-88f0-44bb-94ac-8204b77644f7 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] [instance: f6051269-1d16-42db-8323-67a532dbcf40] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:43:22.583880 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-3925dbd6-88f0-44bb-94ac-8204b77644f7 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:43:22.584620 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-3925dbd6-88f0-44bb-94ac-8204b77644f7 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:43:22.585208 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-3925dbd6-88f0-44bb-94ac-8204b77644f7 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:43:22.586238 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-3925dbd6-88f0-44bb-94ac-8204b77644f7 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:43:22.587257 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-3925dbd6-88f0-44bb-94ac-8204b77644f7 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:43:22.588144 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-3925dbd6-88f0-44bb-94ac-8204b77644f7 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:43:22.766527 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-3925dbd6-88f0-44bb-94ac-8204b77644f7 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd3f0d615187d4779bdd7da8c9f24c796', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0151d58abd047a692f5cc4e7d05bea5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:43:22.775080 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-3925dbd6-88f0-44bb-94ac-8204b77644f7 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] 10.222.0.26 "GET /compute/v2.1/servers/f6051269-1d16-42db-8323-67a532dbcf40" status: 200 len: 1372 microversion: 2.1 time: 0.205559 Oct 19 00:43:22.775666 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 247/362] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:43:22 2022] GET /compute/v2.1/servers/f6051269-1d16-42db-8323-67a532dbcf40 => generated 1372 bytes in 207 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 19 00:43:23.213192 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-035fbe5a-b32d-451c-93a1-0e1ed127bf65 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:43:23.218042 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-035fbe5a-b32d-451c-93a1-0e1ed127bf65 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] [instance: b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:43:23.228350 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-035fbe5a-b32d-451c-93a1-0e1ed127bf65 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:43:23.229484 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-035fbe5a-b32d-451c-93a1-0e1ed127bf65 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:43:23.230108 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-035fbe5a-b32d-451c-93a1-0e1ed127bf65 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:43:23.232135 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-035fbe5a-b32d-451c-93a1-0e1ed127bf65 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:43:23.233073 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-035fbe5a-b32d-451c-93a1-0e1ed127bf65 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:43:23.233741 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-035fbe5a-b32d-451c-93a1-0e1ed127bf65 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:43:23.453228 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-035fbe5a-b32d-451c-93a1-0e1ed127bf65 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:43:23.463814 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-035fbe5a-b32d-451c-93a1-0e1ed127bf65 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:43:23.466660 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-035fbe5a-b32d-451c-93a1-0e1ed127bf65 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:43:23.467003 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-035fbe5a-b32d-451c-93a1-0e1ed127bf65 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] 10.222.0.26 "GET /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2" status: 200 len: 1721 microversion: 2.20 time: 0.263706 Oct 19 00:43:23.467615 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 248/363] 10.222.0.26 () {60 vars in 1277 bytes} [Wed Oct 19 00:43:23 2022] GET /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2 => generated 1721 bytes in 265 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 19 00:43:23.492208 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-9b79be11-e5aa-439b-9d59-bb82dcfe56a3 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:43:23.492824 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-9b79be11-e5aa-439b-9d59-bb82dcfe56a3 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] [instance: 5bdc36c3-1db5-4ff2-808d-1295f314d522] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:43:23.503882 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-9b79be11-e5aa-439b-9d59-bb82dcfe56a3 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:43:23.504380 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-9b79be11-e5aa-439b-9d59-bb82dcfe56a3 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:43:23.504859 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-9b79be11-e5aa-439b-9d59-bb82dcfe56a3 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:43:23.509845 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-9b79be11-e5aa-439b-9d59-bb82dcfe56a3 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:43:23.516159 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-9b79be11-e5aa-439b-9d59-bb82dcfe56a3 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:43:23.516827 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-9b79be11-e5aa-439b-9d59-bb82dcfe56a3 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:43:23.760137 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-9b79be11-e5aa-439b-9d59-bb82dcfe56a3 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '39de6a6fff7c4cc08cc62aa86182d71d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1235d8b054b3405595fca761f08b73ae', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:43:23.776462 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-9b79be11-e5aa-439b-9d59-bb82dcfe56a3 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] 10.222.0.26 "GET /compute/v2.1/servers/5bdc36c3-1db5-4ff2-808d-1295f314d522" status: 200 len: 1319 microversion: 2.1 time: 0.292364 Oct 19 00:43:23.787672 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 249/364] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:43:23 2022] GET /compute/v2.1/servers/5bdc36c3-1db5-4ff2-808d-1295f314d522 => generated 1319 bytes in 319 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:43:23.807457 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-ff5e2c95-3f2b-4681-afe9-1dee577253c9 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:43:23.810068 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-ff5e2c95-3f2b-4681-afe9-1dee577253c9 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] [instance: f6051269-1d16-42db-8323-67a532dbcf40] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:43:23.821573 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ff5e2c95-3f2b-4681-afe9-1dee577253c9 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:43:23.822965 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ff5e2c95-3f2b-4681-afe9-1dee577253c9 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:43:23.824276 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ff5e2c95-3f2b-4681-afe9-1dee577253c9 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:43:23.828081 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ff5e2c95-3f2b-4681-afe9-1dee577253c9 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:43:23.830515 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ff5e2c95-3f2b-4681-afe9-1dee577253c9 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:43:23.831617 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ff5e2c95-3f2b-4681-afe9-1dee577253c9 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:43:24.087086 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-ff5e2c95-3f2b-4681-afe9-1dee577253c9 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd3f0d615187d4779bdd7da8c9f24c796', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0151d58abd047a692f5cc4e7d05bea5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:43:24.098269 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-ff5e2c95-3f2b-4681-afe9-1dee577253c9 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] 10.222.0.26 "GET /compute/v2.1/servers/f6051269-1d16-42db-8323-67a532dbcf40" status: 200 len: 1567 microversion: 2.1 time: 0.301967 Oct 19 00:43:24.099622 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 250/365] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:43:23 2022] GET /compute/v2.1/servers/f6051269-1d16-42db-8323-67a532dbcf40 => generated 1567 bytes in 304 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 19 00:43:24.123289 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-ee588a70-2993-4714-be5a-891ce70f2592 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:43:24.123840 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-ee588a70-2993-4714-be5a-891ce70f2592 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] [instance: f6051269-1d16-42db-8323-67a532dbcf40] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:43:24.126902 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG dbcounter [-] [125641] Writing DB stats nova_api:SELECT=100 {{(pid=125641) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 00:43:24.130954 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ee588a70-2993-4714-be5a-891ce70f2592 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:43:24.131367 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ee588a70-2993-4714-be5a-891ce70f2592 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:43:24.131814 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ee588a70-2993-4714-be5a-891ce70f2592 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:43:24.132682 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ee588a70-2993-4714-be5a-891ce70f2592 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:43:24.133162 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ee588a70-2993-4714-be5a-891ce70f2592 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:43:24.133725 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ee588a70-2993-4714-be5a-891ce70f2592 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:43:24.236108 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-ee588a70-2993-4714-be5a-891ce70f2592 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] 10.222.0.26 "GET /compute/v2.1/servers/f6051269-1d16-42db-8323-67a532dbcf40/os-interface" status: 200 len: 285 microversion: 2.1 time: 0.117589 Oct 19 00:43:24.236897 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 251/366] 10.222.0.26 () {58 vars in 1275 bytes} [Wed Oct 19 00:43:24 2022] GET /compute/v2.1/servers/f6051269-1d16-42db-8323-67a532dbcf40/os-interface => generated 285 bytes in 119 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 19 00:43:24.498466 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-0dad5d5c-3d61-4849-ab62-1d1392b0302c tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:43:24.499236 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-0dad5d5c-3d61-4849-ab62-1d1392b0302c tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] [instance: b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:43:24.508707 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-0dad5d5c-3d61-4849-ab62-1d1392b0302c tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:43:24.509380 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-0dad5d5c-3d61-4849-ab62-1d1392b0302c tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:43:24.509930 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-0dad5d5c-3d61-4849-ab62-1d1392b0302c tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:43:24.510799 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-0dad5d5c-3d61-4849-ab62-1d1392b0302c tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:43:24.511374 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-0dad5d5c-3d61-4849-ab62-1d1392b0302c tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:43:24.511902 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-0dad5d5c-3d61-4849-ab62-1d1392b0302c tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:43:24.806245 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-0dad5d5c-3d61-4849-ab62-1d1392b0302c tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:43:24.815837 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-0dad5d5c-3d61-4849-ab62-1d1392b0302c tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:43:24.817578 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-0dad5d5c-3d61-4849-ab62-1d1392b0302c tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:43:24.818780 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-0dad5d5c-3d61-4849-ab62-1d1392b0302c tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] 10.222.0.26 "GET /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2" status: 200 len: 1724 microversion: 2.20 time: 0.325655 Oct 19 00:43:24.831136 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 252/367] 10.222.0.26 () {60 vars in 1277 bytes} [Wed Oct 19 00:43:24 2022] GET /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2 => generated 1724 bytes in 338 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 19 00:43:24.847986 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-e647f6fe-59f4-468b-89cc-e711a03db86a tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:43:24.848685 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-e647f6fe-59f4-468b-89cc-e711a03db86a tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] [instance: 5bdc36c3-1db5-4ff2-808d-1295f314d522] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:43:24.855725 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-e647f6fe-59f4-468b-89cc-e711a03db86a tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:43:24.856719 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-e647f6fe-59f4-468b-89cc-e711a03db86a tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:43:24.857412 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-e647f6fe-59f4-468b-89cc-e711a03db86a tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:43:24.858351 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-e647f6fe-59f4-468b-89cc-e711a03db86a tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:43:24.858926 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-e647f6fe-59f4-468b-89cc-e711a03db86a tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:43:24.859404 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-e647f6fe-59f4-468b-89cc-e711a03db86a tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:43:25.113912 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-e647f6fe-59f4-468b-89cc-e711a03db86a tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '39de6a6fff7c4cc08cc62aa86182d71d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1235d8b054b3405595fca761f08b73ae', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:43:25.123948 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-e647f6fe-59f4-468b-89cc-e711a03db86a tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] 10.222.0.26 "GET /compute/v2.1/servers/5bdc36c3-1db5-4ff2-808d-1295f314d522" status: 200 len: 1319 microversion: 2.1 time: 0.291270 Oct 19 00:43:25.134592 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 253/368] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:43:24 2022] GET /compute/v2.1/servers/5bdc36c3-1db5-4ff2-808d-1295f314d522 => generated 1319 bytes in 293 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:43:25.134592 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-3ca49fb3-57d9-42b2-955c-92e4e68e517c tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "b8d00552-fcdb-4fc2-b489-3a2cb14e89e7"}} {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 19 00:43:25.134592 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-3ca49fb3-57d9-42b2-955c-92e4e68e517c tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] [instance: b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:43:25.140978 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-3ca49fb3-57d9-42b2-955c-92e4e68e517c tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:43:25.141633 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-3ca49fb3-57d9-42b2-955c-92e4e68e517c tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:43:25.142164 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-3ca49fb3-57d9-42b2-955c-92e4e68e517c tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:43:25.143012 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-3ca49fb3-57d9-42b2-955c-92e4e68e517c tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:43:25.143566 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-3ca49fb3-57d9-42b2-955c-92e4e68e517c tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:43:25.144082 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-3ca49fb3-57d9-42b2-955c-92e4e68e517c tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:43:25.531608 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-3ca49fb3-57d9-42b2-955c-92e4e68e517c tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] 10.222.0.26 "POST /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2/os-volume_attachments" status: 200 len: 188 microversion: 2.20 time: 0.405426 Oct 19 00:43:25.532256 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 254/369] 10.222.0.26 () {62 vars in 1364 bytes} [Wed Oct 19 00:43:25 2022] POST /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2/os-volume_attachments => generated 188 bytes in 407 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Oct 19 00:43:25.630072 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [req-e822b888-9788-4d30-8f0e-8e61cecd84a9 req-248dd028-381c-4ea3-92ff-a62b8d5c6477 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "f6051269-1d16-42db-8323-67a532dbcf40", "tag": "667c66be-7422-49b6-8e96-6c36f067b697"}]} {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 19 00:43:25.642318 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-e822b888-9788-4d30-8f0e-8e61cecd84a9 req-248dd028-381c-4ea3-92ff-a62b8d5c6477 service nova] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:43:25.643250 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-e822b888-9788-4d30-8f0e-8e61cecd84a9 req-248dd028-381c-4ea3-92ff-a62b8d5c6477 service nova] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:43:25.643919 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-e822b888-9788-4d30-8f0e-8e61cecd84a9 req-248dd028-381c-4ea3-92ff-a62b8d5c6477 service nova] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:43:25.657443 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.compute.server_external_events [req-e822b888-9788-4d30-8f0e-8e61cecd84a9 req-248dd028-381c-4ea3-92ff-a62b8d5c6477 service nova] Creating event network-changed:667c66be-7422-49b6-8e96-6c36f067b697 for instance f6051269-1d16-42db-8323-67a532dbcf40 on hv-cinder-maste Oct 19 00:43:25.662751 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [req-e822b888-9788-4d30-8f0e-8e61cecd84a9 req-248dd028-381c-4ea3-92ff-a62b8d5c6477 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.037338 Oct 19 00:43:25.663607 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 255/370] 10.222.0.26 () {64 vars in 1317 bytes} [Wed Oct 19 00:43:25 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 19 00:43:25.748874 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-559ee7c5-9d0f-4762-ba2a-5e45d438198f tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Action: 'action', calling method: >, body: {"unshelve": null} {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 19 00:43:25.749424 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-559ee7c5-9d0f-4762-ba2a-5e45d438198f tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] [instance: b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:43:25.756691 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-559ee7c5-9d0f-4762-ba2a-5e45d438198f tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:43:25.757327 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-559ee7c5-9d0f-4762-ba2a-5e45d438198f tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:43:25.758034 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-559ee7c5-9d0f-4762-ba2a-5e45d438198f tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:43:25.759450 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-559ee7c5-9d0f-4762-ba2a-5e45d438198f tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:43:25.759450 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-559ee7c5-9d0f-4762-ba2a-5e45d438198f tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:43:25.760001 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-559ee7c5-9d0f-4762-ba2a-5e45d438198f tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:43:25.815944 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-559ee7c5-9d0f-4762-ba2a-5e45d438198f tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] [instance: b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2] Unshelving instance with old availability_zone "None" to new availability_zone "not provided" and host "None". {{(pid=125641) unshelve /opt/stack/nova/nova/compute/api.py:4593}} Oct 19 00:43:25.907854 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-559ee7c5-9d0f-4762-ba2a-5e45d438198f tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] 10.222.0.26 "POST /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2/action" status: 202 len: 0 microversion: 2.20 time: 0.163332 Oct 19 00:43:25.908965 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 256/371] 10.222.0.26 () {62 vars in 1319 bytes} [Wed Oct 19 00:43:25 2022] POST /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2/action => generated 0 bytes in 166 msecs (HTTP/1.1 202) 9 headers in 363 bytes (2 switches on core 0) Oct 19 00:43:25.917215 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-93d20364-a0fa-4139-9f6e-c952a73a95be tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:43:25.917850 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-93d20364-a0fa-4139-9f6e-c952a73a95be tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] [instance: b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:43:25.926309 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-93d20364-a0fa-4139-9f6e-c952a73a95be tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:43:25.926994 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-93d20364-a0fa-4139-9f6e-c952a73a95be tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:43:25.927680 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-93d20364-a0fa-4139-9f6e-c952a73a95be tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:43:25.928668 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-93d20364-a0fa-4139-9f6e-c952a73a95be tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:43:25.929234 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-93d20364-a0fa-4139-9f6e-c952a73a95be tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:43:25.929795 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-93d20364-a0fa-4139-9f6e-c952a73a95be tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:43:26.132200 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-93d20364-a0fa-4139-9f6e-c952a73a95be tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:43:26.145778 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-93d20364-a0fa-4139-9f6e-c952a73a95be tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:43:26.152065 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-93d20364-a0fa-4139-9f6e-c952a73a95be tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:43:26.156478 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-93d20364-a0fa-4139-9f6e-c952a73a95be tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] 10.222.0.26 "GET /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2" status: 200 len: 1810 microversion: 2.20 time: 0.243388 Oct 19 00:43:26.157192 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 257/372] 10.222.0.26 () {60 vars in 1277 bytes} [Wed Oct 19 00:43:25 2022] GET /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2 => generated 1810 bytes in 245 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 19 00:43:26.166370 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-3410358f-3b03-4ea9-a0c6-a34b7a6aa5d3 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:43:26.167027 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-3410358f-3b03-4ea9-a0c6-a34b7a6aa5d3 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] [instance: 5bdc36c3-1db5-4ff2-808d-1295f314d522] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:43:26.174317 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-3410358f-3b03-4ea9-a0c6-a34b7a6aa5d3 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:43:26.174883 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-3410358f-3b03-4ea9-a0c6-a34b7a6aa5d3 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:43:26.175345 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-3410358f-3b03-4ea9-a0c6-a34b7a6aa5d3 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:43:26.176117 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-3410358f-3b03-4ea9-a0c6-a34b7a6aa5d3 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:43:26.176580 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-3410358f-3b03-4ea9-a0c6-a34b7a6aa5d3 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:43:26.177040 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-3410358f-3b03-4ea9-a0c6-a34b7a6aa5d3 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:43:26.432802 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-3410358f-3b03-4ea9-a0c6-a34b7a6aa5d3 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '39de6a6fff7c4cc08cc62aa86182d71d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1235d8b054b3405595fca761f08b73ae', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:43:26.446366 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-3410358f-3b03-4ea9-a0c6-a34b7a6aa5d3 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] 10.222.0.26 "GET /compute/v2.1/servers/5bdc36c3-1db5-4ff2-808d-1295f314d522" status: 200 len: 1319 microversion: 2.1 time: 0.287878 Oct 19 00:43:26.447352 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 258/373] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:43:26 2022] GET /compute/v2.1/servers/5bdc36c3-1db5-4ff2-808d-1295f314d522 => generated 1319 bytes in 290 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:43:27.213993 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-c077c21e-e821-4fc8-a537-2ef02241d93d tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:43:27.215765 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-c077c21e-e821-4fc8-a537-2ef02241d93d tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] [instance: b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:43:27.231913 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-c077c21e-e821-4fc8-a537-2ef02241d93d tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:43:27.234839 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-c077c21e-e821-4fc8-a537-2ef02241d93d tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:43:27.235553 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-c077c21e-e821-4fc8-a537-2ef02241d93d tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:43:27.238209 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-c077c21e-e821-4fc8-a537-2ef02241d93d tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:43:27.239084 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-c077c21e-e821-4fc8-a537-2ef02241d93d tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:43:27.241026 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-c077c21e-e821-4fc8-a537-2ef02241d93d tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:43:27.587463 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-c077c21e-e821-4fc8-a537-2ef02241d93d tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:43:27.593085 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG dbcounter [-] [125641] Writing DB stats nova_cell1:SELECT=91,nova_cell1:INSERT=4,nova_cell1:UPDATE=5 {{(pid=125641) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 00:43:27.604686 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-c077c21e-e821-4fc8-a537-2ef02241d93d tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:43:27.608774 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-c077c21e-e821-4fc8-a537-2ef02241d93d tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:43:27.610047 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-c077c21e-e821-4fc8-a537-2ef02241d93d tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] 10.222.0.26 "GET /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2" status: 200 len: 1750 microversion: 2.20 time: 0.402400 Oct 19 00:43:27.622541 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 259/374] 10.222.0.26 () {60 vars in 1277 bytes} [Wed Oct 19 00:43:27 2022] GET /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2 => generated 1750 bytes in 404 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 19 00:43:27.635703 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [req-744cfea6-60e9-44b7-985a-67541e90ed92 req-6dcf2cee-331d-4b39-8379-3815febbaa19 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "5bdc36c3-1db5-4ff2-808d-1295f314d522", "name": "network-vif-plugged", "status": "completed", "tag": "97ea2bef-8a45-4c17-8415-e62d303621cb"}]} {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 19 00:43:27.650086 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-744cfea6-60e9-44b7-985a-67541e90ed92 req-6dcf2cee-331d-4b39-8379-3815febbaa19 service nova] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:43:27.650591 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-744cfea6-60e9-44b7-985a-67541e90ed92 req-6dcf2cee-331d-4b39-8379-3815febbaa19 service nova] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:43:27.651035 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-744cfea6-60e9-44b7-985a-67541e90ed92 req-6dcf2cee-331d-4b39-8379-3815febbaa19 service nova] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:43:27.665869 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.compute.server_external_events [req-744cfea6-60e9-44b7-985a-67541e90ed92 req-6dcf2cee-331d-4b39-8379-3815febbaa19 service nova] Creating event network-vif-plugged:97ea2bef-8a45-4c17-8415-e62d303621cb for instance 5bdc36c3-1db5-4ff2-808d-1295f314d522 on hv-cinder-maste Oct 19 00:43:27.677512 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [req-744cfea6-60e9-44b7-985a-67541e90ed92 req-6dcf2cee-331d-4b39-8379-3815febbaa19 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.047343 Oct 19 00:43:27.678666 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 260/375] 10.222.0.26 () {64 vars in 1317 bytes} [Wed Oct 19 00:43:27 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 67 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 19 00:43:27.686694 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-22b04f22-d2e1-4e2c-b2ae-f50c65f483b4 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:43:27.687572 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-22b04f22-d2e1-4e2c-b2ae-f50c65f483b4 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] [instance: 5bdc36c3-1db5-4ff2-808d-1295f314d522] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:43:27.695705 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-22b04f22-d2e1-4e2c-b2ae-f50c65f483b4 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:43:27.696272 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-22b04f22-d2e1-4e2c-b2ae-f50c65f483b4 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:43:27.696716 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-22b04f22-d2e1-4e2c-b2ae-f50c65f483b4 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:43:27.697594 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-22b04f22-d2e1-4e2c-b2ae-f50c65f483b4 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:43:27.698130 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-22b04f22-d2e1-4e2c-b2ae-f50c65f483b4 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:43:27.698607 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-22b04f22-d2e1-4e2c-b2ae-f50c65f483b4 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:43:28.065536 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-22b04f22-d2e1-4e2c-b2ae-f50c65f483b4 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '39de6a6fff7c4cc08cc62aa86182d71d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1235d8b054b3405595fca761f08b73ae', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:43:28.094079 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-22b04f22-d2e1-4e2c-b2ae-f50c65f483b4 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] 10.222.0.26 "GET /compute/v2.1/servers/5bdc36c3-1db5-4ff2-808d-1295f314d522" status: 200 len: 1319 microversion: 2.1 time: 0.412681 Oct 19 00:43:28.095714 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 261/376] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:43:27 2022] GET /compute/v2.1/servers/5bdc36c3-1db5-4ff2-808d-1295f314d522 => generated 1319 bytes in 416 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:43:28.639346 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-91c38ca1-b266-441c-8598-0aba3639a132 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:43:28.641138 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-91c38ca1-b266-441c-8598-0aba3639a132 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] [instance: b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:43:28.649017 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-91c38ca1-b266-441c-8598-0aba3639a132 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:43:28.649853 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-91c38ca1-b266-441c-8598-0aba3639a132 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:43:28.650300 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-91c38ca1-b266-441c-8598-0aba3639a132 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:43:28.651161 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-91c38ca1-b266-441c-8598-0aba3639a132 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:43:28.651669 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-91c38ca1-b266-441c-8598-0aba3639a132 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:43:28.652175 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-91c38ca1-b266-441c-8598-0aba3639a132 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:43:28.804644 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-91c38ca1-b266-441c-8598-0aba3639a132 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:43:28.815436 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-91c38ca1-b266-441c-8598-0aba3639a132 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:43:28.816892 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-91c38ca1-b266-441c-8598-0aba3639a132 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:43:28.818199 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-91c38ca1-b266-441c-8598-0aba3639a132 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] 10.222.0.26 "GET /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2" status: 200 len: 1752 microversion: 2.20 time: 0.183902 Oct 19 00:43:28.829086 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 262/377] 10.222.0.26 () {60 vars in 1277 bytes} [Wed Oct 19 00:43:28 2022] GET /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2 => generated 1752 bytes in 195 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 19 00:43:29.120836 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-739b0721-a334-4d2d-9ded-d8fa777b615e tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:43:29.121631 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-739b0721-a334-4d2d-9ded-d8fa777b615e tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] [instance: 5bdc36c3-1db5-4ff2-808d-1295f314d522] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:43:29.127924 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-739b0721-a334-4d2d-9ded-d8fa777b615e tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:43:29.128403 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-739b0721-a334-4d2d-9ded-d8fa777b615e tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:43:29.128827 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-739b0721-a334-4d2d-9ded-d8fa777b615e tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:43:29.129670 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-739b0721-a334-4d2d-9ded-d8fa777b615e tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:43:29.130034 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-739b0721-a334-4d2d-9ded-d8fa777b615e tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:43:29.130512 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-739b0721-a334-4d2d-9ded-d8fa777b615e tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:43:29.296934 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-739b0721-a334-4d2d-9ded-d8fa777b615e tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '39de6a6fff7c4cc08cc62aa86182d71d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1235d8b054b3405595fca761f08b73ae', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:43:29.306766 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-739b0721-a334-4d2d-9ded-d8fa777b615e tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] 10.222.0.26 "GET /compute/v2.1/servers/5bdc36c3-1db5-4ff2-808d-1295f314d522" status: 200 len: 1319 microversion: 2.1 time: 0.190425 Oct 19 00:43:29.307939 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 263/378] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:43:29 2022] GET /compute/v2.1/servers/5bdc36c3-1db5-4ff2-808d-1295f314d522 => generated 1319 bytes in 193 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 19 00:43:29.707813 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [req-2de97003-2449-4faa-8684-74a51e473db5 req-6534f656-888b-41f4-a045-afd43831082a service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2", "name": "network-vif-unplugged", "status": "completed", "tag": "6aea38b4-b7a0-4e8e-82dc-deb19c260092"}]} {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 19 00:43:29.728561 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-2de97003-2449-4faa-8684-74a51e473db5 req-6534f656-888b-41f4-a045-afd43831082a service nova] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:43:29.728834 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-2de97003-2449-4faa-8684-74a51e473db5 req-6534f656-888b-41f4-a045-afd43831082a service nova] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:43:29.729444 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-2de97003-2449-4faa-8684-74a51e473db5 req-6534f656-888b-41f4-a045-afd43831082a service nova] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:43:29.742877 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.compute.server_external_events [req-2de97003-2449-4faa-8684-74a51e473db5 req-6534f656-888b-41f4-a045-afd43831082a service nova] Creating event network-vif-unplugged:6aea38b4-b7a0-4e8e-82dc-deb19c260092 for instance b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2 on hv-cinder-maste Oct 19 00:43:29.748954 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [req-2de97003-2449-4faa-8684-74a51e473db5 req-6534f656-888b-41f4-a045-afd43831082a service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.044754 Oct 19 00:43:29.749550 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 264/379] 10.222.0.26 () {64 vars in 1317 bytes} [Wed Oct 19 00:43:29 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 19 00:43:29.847853 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-22108178-da34-4869-80db-aef28370a68f tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:43:29.849186 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-22108178-da34-4869-80db-aef28370a68f tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] [instance: b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:43:29.862021 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-22108178-da34-4869-80db-aef28370a68f tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:43:29.862568 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-22108178-da34-4869-80db-aef28370a68f tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:43:29.866838 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-22108178-da34-4869-80db-aef28370a68f tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:43:29.867856 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-22108178-da34-4869-80db-aef28370a68f tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:43:29.868519 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-22108178-da34-4869-80db-aef28370a68f tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:43:29.869267 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-22108178-da34-4869-80db-aef28370a68f tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:43:30.052581 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-22108178-da34-4869-80db-aef28370a68f tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:43:30.065715 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-22108178-da34-4869-80db-aef28370a68f tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:43:30.067517 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-22108178-da34-4869-80db-aef28370a68f tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:43:30.069135 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-22108178-da34-4869-80db-aef28370a68f tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] 10.222.0.26 "GET /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2" status: 200 len: 1808 microversion: 2.20 time: 0.226948 Oct 19 00:43:30.070223 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 265/380] 10.222.0.26 () {60 vars in 1277 bytes} [Wed Oct 19 00:43:29 2022] GET /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2 => generated 1808 bytes in 229 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 19 00:43:30.331958 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-f1631306-fca1-4674-ba12-3f91c840aaf6 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:43:30.332999 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-f1631306-fca1-4674-ba12-3f91c840aaf6 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] [instance: 5bdc36c3-1db5-4ff2-808d-1295f314d522] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:43:30.340212 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-f1631306-fca1-4674-ba12-3f91c840aaf6 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:43:30.341457 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-f1631306-fca1-4674-ba12-3f91c840aaf6 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:43:30.342219 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-f1631306-fca1-4674-ba12-3f91c840aaf6 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:43:30.343023 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-f1631306-fca1-4674-ba12-3f91c840aaf6 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:43:30.343714 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-f1631306-fca1-4674-ba12-3f91c840aaf6 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:43:30.344194 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-f1631306-fca1-4674-ba12-3f91c840aaf6 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:43:30.499015 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-f1631306-fca1-4674-ba12-3f91c840aaf6 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '39de6a6fff7c4cc08cc62aa86182d71d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1235d8b054b3405595fca761f08b73ae', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:43:30.517272 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-f1631306-fca1-4674-ba12-3f91c840aaf6 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] 10.222.0.26 "GET /compute/v2.1/servers/5bdc36c3-1db5-4ff2-808d-1295f314d522" status: 200 len: 1319 microversion: 2.1 time: 0.189297 Oct 19 00:43:30.517881 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 266/381] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:43:30 2022] GET /compute/v2.1/servers/5bdc36c3-1db5-4ff2-808d-1295f314d522 => generated 1319 bytes in 191 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 19 00:43:31.096885 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-9cdc384d-7031-40c9-bdb1-a788fc324ba5 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:43:31.098535 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-9cdc384d-7031-40c9-bdb1-a788fc324ba5 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] [instance: b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:43:31.105710 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-9cdc384d-7031-40c9-bdb1-a788fc324ba5 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:43:31.107160 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-9cdc384d-7031-40c9-bdb1-a788fc324ba5 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:43:31.107828 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-9cdc384d-7031-40c9-bdb1-a788fc324ba5 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:43:31.108875 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-9cdc384d-7031-40c9-bdb1-a788fc324ba5 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:43:31.109597 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-9cdc384d-7031-40c9-bdb1-a788fc324ba5 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:43:31.110260 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-9cdc384d-7031-40c9-bdb1-a788fc324ba5 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:43:31.285893 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-9cdc384d-7031-40c9-bdb1-a788fc324ba5 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:43:31.298159 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-9cdc384d-7031-40c9-bdb1-a788fc324ba5 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:43:31.300161 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-9cdc384d-7031-40c9-bdb1-a788fc324ba5 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:43:31.301831 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-9cdc384d-7031-40c9-bdb1-a788fc324ba5 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] 10.222.0.26 "GET /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2" status: 200 len: 1808 microversion: 2.20 time: 0.209668 Oct 19 00:43:31.302680 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 267/382] 10.222.0.26 () {60 vars in 1277 bytes} [Wed Oct 19 00:43:31 2022] GET /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2 => generated 1808 bytes in 211 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 19 00:43:31.560152 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-048b115d-d539-489c-91e0-689c3aeb2c28 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:43:31.563634 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-048b115d-d539-489c-91e0-689c3aeb2c28 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] [instance: 5bdc36c3-1db5-4ff2-808d-1295f314d522] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:43:31.572038 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-048b115d-d539-489c-91e0-689c3aeb2c28 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:43:31.572519 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-048b115d-d539-489c-91e0-689c3aeb2c28 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:43:31.572950 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-048b115d-d539-489c-91e0-689c3aeb2c28 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:43:31.573727 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-048b115d-d539-489c-91e0-689c3aeb2c28 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:43:31.575871 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-048b115d-d539-489c-91e0-689c3aeb2c28 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:43:31.576309 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-048b115d-d539-489c-91e0-689c3aeb2c28 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:43:31.734371 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-048b115d-d539-489c-91e0-689c3aeb2c28 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '39de6a6fff7c4cc08cc62aa86182d71d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1235d8b054b3405595fca761f08b73ae', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:43:31.743229 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-048b115d-d539-489c-91e0-689c3aeb2c28 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] 10.222.0.26 "GET /compute/v2.1/servers/5bdc36c3-1db5-4ff2-808d-1295f314d522" status: 200 len: 1319 microversion: 2.1 time: 0.201767 Oct 19 00:43:31.751134 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 268/383] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:43:31 2022] GET /compute/v2.1/servers/5bdc36c3-1db5-4ff2-808d-1295f314d522 => generated 1319 bytes in 203 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:43:32.331646 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-ad3d9b8b-3e55-41b9-a803-386dc08d3d83 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:43:32.333688 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-ad3d9b8b-3e55-41b9-a803-386dc08d3d83 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] [instance: b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:43:32.341875 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ad3d9b8b-3e55-41b9-a803-386dc08d3d83 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:43:32.342770 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ad3d9b8b-3e55-41b9-a803-386dc08d3d83 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:43:32.343315 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ad3d9b8b-3e55-41b9-a803-386dc08d3d83 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:43:32.344141 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ad3d9b8b-3e55-41b9-a803-386dc08d3d83 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:43:32.344738 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ad3d9b8b-3e55-41b9-a803-386dc08d3d83 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:43:32.345496 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ad3d9b8b-3e55-41b9-a803-386dc08d3d83 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:43:32.763462 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-ad3d9b8b-3e55-41b9-a803-386dc08d3d83 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:43:32.773995 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-ad3d9b8b-3e55-41b9-a803-386dc08d3d83 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:43:32.775483 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-ad3d9b8b-3e55-41b9-a803-386dc08d3d83 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:43:32.776686 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-ad3d9b8b-3e55-41b9-a803-386dc08d3d83 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] 10.222.0.26 "GET /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2" status: 200 len: 1808 microversion: 2.20 time: 0.450635 Oct 19 00:43:32.777243 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 269/384] 10.222.0.26 () {60 vars in 1277 bytes} [Wed Oct 19 00:43:32 2022] GET /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2 => generated 1808 bytes in 452 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 19 00:43:32.788217 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-b1f0ab50-1eaa-4689-b85b-4ecabf9cbd45 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:43:32.789489 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-b1f0ab50-1eaa-4689-b85b-4ecabf9cbd45 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] [instance: 5bdc36c3-1db5-4ff2-808d-1295f314d522] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:43:32.796421 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-b1f0ab50-1eaa-4689-b85b-4ecabf9cbd45 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:43:32.797194 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-b1f0ab50-1eaa-4689-b85b-4ecabf9cbd45 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:43:32.797690 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-b1f0ab50-1eaa-4689-b85b-4ecabf9cbd45 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:43:32.798647 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-b1f0ab50-1eaa-4689-b85b-4ecabf9cbd45 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:43:32.799273 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-b1f0ab50-1eaa-4689-b85b-4ecabf9cbd45 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:43:32.799704 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-b1f0ab50-1eaa-4689-b85b-4ecabf9cbd45 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:43:32.976106 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-b1f0ab50-1eaa-4689-b85b-4ecabf9cbd45 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '39de6a6fff7c4cc08cc62aa86182d71d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1235d8b054b3405595fca761f08b73ae', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:43:32.987473 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-b1f0ab50-1eaa-4689-b85b-4ecabf9cbd45 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] 10.222.0.26 "GET /compute/v2.1/servers/5bdc36c3-1db5-4ff2-808d-1295f314d522" status: 200 len: 1319 microversion: 2.1 time: 0.208462 Oct 19 00:43:32.990780 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 270/385] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:43:32 2022] GET /compute/v2.1/servers/5bdc36c3-1db5-4ff2-808d-1295f314d522 => generated 1319 bytes in 211 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:43:33.807412 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-ac04f523-7f27-4467-93dc-176c0be01ca2 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:43:33.808213 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-ac04f523-7f27-4467-93dc-176c0be01ca2 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] [instance: b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:43:33.839217 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ac04f523-7f27-4467-93dc-176c0be01ca2 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:43:33.844631 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ac04f523-7f27-4467-93dc-176c0be01ca2 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:43:33.846530 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ac04f523-7f27-4467-93dc-176c0be01ca2 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:43:33.849626 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ac04f523-7f27-4467-93dc-176c0be01ca2 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:43:33.850952 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ac04f523-7f27-4467-93dc-176c0be01ca2 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:43:33.852147 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ac04f523-7f27-4467-93dc-176c0be01ca2 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:43:34.140354 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-ac04f523-7f27-4467-93dc-176c0be01ca2 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:43:34.154428 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-ac04f523-7f27-4467-93dc-176c0be01ca2 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:43:34.156673 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-ac04f523-7f27-4467-93dc-176c0be01ca2 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:43:34.158185 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-ac04f523-7f27-4467-93dc-176c0be01ca2 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] 10.222.0.26 "GET /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2" status: 200 len: 1808 microversion: 2.20 time: 0.354801 Oct 19 00:43:34.178576 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 271/386] 10.222.0.26 () {60 vars in 1277 bytes} [Wed Oct 19 00:43:33 2022] GET /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2 => generated 1808 bytes in 371 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 19 00:43:34.206525 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-23ae01f8-c478-44f6-9a26-6a9d206bf055 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:43:34.206525 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-23ae01f8-c478-44f6-9a26-6a9d206bf055 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] [instance: 5bdc36c3-1db5-4ff2-808d-1295f314d522] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:43:34.223382 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-23ae01f8-c478-44f6-9a26-6a9d206bf055 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:43:34.225062 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-23ae01f8-c478-44f6-9a26-6a9d206bf055 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:43:34.226058 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-23ae01f8-c478-44f6-9a26-6a9d206bf055 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:43:34.227089 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-23ae01f8-c478-44f6-9a26-6a9d206bf055 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:43:34.227580 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-23ae01f8-c478-44f6-9a26-6a9d206bf055 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:43:34.228010 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-23ae01f8-c478-44f6-9a26-6a9d206bf055 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:43:34.533057 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-23ae01f8-c478-44f6-9a26-6a9d206bf055 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '39de6a6fff7c4cc08cc62aa86182d71d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1235d8b054b3405595fca761f08b73ae', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:43:34.541789 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-23ae01f8-c478-44f6-9a26-6a9d206bf055 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] 10.222.0.26 "GET /compute/v2.1/servers/5bdc36c3-1db5-4ff2-808d-1295f314d522" status: 200 len: 1508 microversion: 2.1 time: 0.360940 Oct 19 00:43:34.542456 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 272/387] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:43:34 2022] GET /compute/v2.1/servers/5bdc36c3-1db5-4ff2-808d-1295f314d522 => generated 1508 bytes in 363 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:43:35.195861 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-07573c2f-25ee-44ea-ad70-15a0fc8152ef tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:43:35.196555 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-07573c2f-25ee-44ea-ad70-15a0fc8152ef tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] [instance: b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:43:35.203377 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-07573c2f-25ee-44ea-ad70-15a0fc8152ef tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:43:35.203871 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-07573c2f-25ee-44ea-ad70-15a0fc8152ef tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:43:35.204310 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-07573c2f-25ee-44ea-ad70-15a0fc8152ef tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:43:35.205025 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-07573c2f-25ee-44ea-ad70-15a0fc8152ef tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:43:35.205480 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-07573c2f-25ee-44ea-ad70-15a0fc8152ef tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:43:35.205914 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-07573c2f-25ee-44ea-ad70-15a0fc8152ef tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:43:35.357087 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-07573c2f-25ee-44ea-ad70-15a0fc8152ef tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:43:35.366625 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-07573c2f-25ee-44ea-ad70-15a0fc8152ef tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:43:35.368085 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-07573c2f-25ee-44ea-ad70-15a0fc8152ef tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:43:35.369770 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-07573c2f-25ee-44ea-ad70-15a0fc8152ef tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] 10.222.0.26 "GET /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2" status: 200 len: 1808 microversion: 2.20 time: 0.177638 Oct 19 00:43:35.380477 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 273/388] 10.222.0.26 () {60 vars in 1277 bytes} [Wed Oct 19 00:43:35 2022] GET /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2 => generated 1808 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 19 00:43:36.397137 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-503a71be-457b-4fe1-b2eb-7728fa9d13b0 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:43:36.397801 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-503a71be-457b-4fe1-b2eb-7728fa9d13b0 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] [instance: b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:43:36.404901 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-503a71be-457b-4fe1-b2eb-7728fa9d13b0 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:43:36.405394 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-503a71be-457b-4fe1-b2eb-7728fa9d13b0 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:43:36.405913 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-503a71be-457b-4fe1-b2eb-7728fa9d13b0 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:43:36.406690 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-503a71be-457b-4fe1-b2eb-7728fa9d13b0 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:43:36.407142 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-503a71be-457b-4fe1-b2eb-7728fa9d13b0 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:43:36.407566 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-503a71be-457b-4fe1-b2eb-7728fa9d13b0 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:43:36.562931 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-503a71be-457b-4fe1-b2eb-7728fa9d13b0 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:43:36.572228 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-503a71be-457b-4fe1-b2eb-7728fa9d13b0 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:43:36.573669 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-503a71be-457b-4fe1-b2eb-7728fa9d13b0 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:43:36.575829 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-503a71be-457b-4fe1-b2eb-7728fa9d13b0 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] 10.222.0.26 "GET /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2" status: 200 len: 1808 microversion: 2.20 time: 0.181857 Oct 19 00:43:36.576285 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 274/389] 10.222.0.26 () {60 vars in 1277 bytes} [Wed Oct 19 00:43:36 2022] GET /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2 => generated 1808 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 19 00:43:37.601701 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-19ad8e08-773b-4072-82a1-c2e02dbfa39b tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:43:37.602407 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-19ad8e08-773b-4072-82a1-c2e02dbfa39b tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] [instance: b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:43:37.608736 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-19ad8e08-773b-4072-82a1-c2e02dbfa39b tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:43:37.609224 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-19ad8e08-773b-4072-82a1-c2e02dbfa39b tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:43:37.609662 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-19ad8e08-773b-4072-82a1-c2e02dbfa39b tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:43:37.610366 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-19ad8e08-773b-4072-82a1-c2e02dbfa39b tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:43:37.610855 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-19ad8e08-773b-4072-82a1-c2e02dbfa39b tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:43:37.611281 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-19ad8e08-773b-4072-82a1-c2e02dbfa39b tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:43:37.751282 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-19ad8e08-773b-4072-82a1-c2e02dbfa39b tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:43:37.761539 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-19ad8e08-773b-4072-82a1-c2e02dbfa39b tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:43:37.763712 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-19ad8e08-773b-4072-82a1-c2e02dbfa39b tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:43:37.765037 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-19ad8e08-773b-4072-82a1-c2e02dbfa39b tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] 10.222.0.26 "GET /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2" status: 200 len: 1808 microversion: 2.20 time: 0.167204 Oct 19 00:43:37.765909 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 275/390] 10.222.0.26 () {60 vars in 1277 bytes} [Wed Oct 19 00:43:37 2022] GET /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2 => generated 1808 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 19 00:43:37.951672 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-0f461f36-84df-4099-9549-730c592a84a0 tempest-TestShelveInstance-770582455 tempest-TestShelveInstance-770582455-project] Calling method '.version_select at 0x7f71c81f4940>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:43:37.983828 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-0f461f36-84df-4099-9549-730c592a84a0 tempest-TestShelveInstance-770582455 tempest-TestShelveInstance-770582455-project] 10.222.0.26 "DELETE /compute/v2.1/os-keypairs/tempest-TestShelveInstance-480977406" status: 202 len: 0 microversion: 2.1 time: 0.037084 Oct 19 00:43:37.984746 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 276/391] 10.222.0.26 () {58 vars in 1251 bytes} [Wed Oct 19 00:43:37 2022] DELETE /compute/v2.1/os-keypairs/tempest-TestShelveInstance-480977406 => generated 0 bytes in 39 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Oct 19 00:43:38.792755 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-1aed72c5-6780-4b4e-a121-070d1d612acd tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:43:38.793530 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-1aed72c5-6780-4b4e-a121-070d1d612acd tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] [instance: b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:43:38.800077 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-1aed72c5-6780-4b4e-a121-070d1d612acd tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:43:38.800566 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-1aed72c5-6780-4b4e-a121-070d1d612acd tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:43:38.801001 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-1aed72c5-6780-4b4e-a121-070d1d612acd tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:43:38.801716 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-1aed72c5-6780-4b4e-a121-070d1d612acd tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:43:38.802170 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-1aed72c5-6780-4b4e-a121-070d1d612acd tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:43:38.802620 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-1aed72c5-6780-4b4e-a121-070d1d612acd tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:43:39.028854 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-1aed72c5-6780-4b4e-a121-070d1d612acd tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:43:39.039793 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-1aed72c5-6780-4b4e-a121-070d1d612acd tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:43:39.041276 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-1aed72c5-6780-4b4e-a121-070d1d612acd tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:43:39.042539 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-1aed72c5-6780-4b4e-a121-070d1d612acd tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] 10.222.0.26 "GET /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2" status: 200 len: 1808 microversion: 2.20 time: 0.253741 Oct 19 00:43:39.043206 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 277/392] 10.222.0.26 () {60 vars in 1277 bytes} [Wed Oct 19 00:43:38 2022] GET /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2 => generated 1808 bytes in 255 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 19 00:43:40.085867 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-71867d7d-cddb-460e-85ed-0245ad1b4cc4 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:43:40.086734 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-71867d7d-cddb-460e-85ed-0245ad1b4cc4 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] [instance: b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:43:40.132022 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-71867d7d-cddb-460e-85ed-0245ad1b4cc4 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:43:40.133014 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-71867d7d-cddb-460e-85ed-0245ad1b4cc4 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.016s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:43:40.133885 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-71867d7d-cddb-460e-85ed-0245ad1b4cc4 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:43:40.135434 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-71867d7d-cddb-460e-85ed-0245ad1b4cc4 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:43:40.136038 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-71867d7d-cddb-460e-85ed-0245ad1b4cc4 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:43:40.142529 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-71867d7d-cddb-460e-85ed-0245ad1b4cc4 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:43:40.734744 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-71867d7d-cddb-460e-85ed-0245ad1b4cc4 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:43:40.740292 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG dbcounter [-] [125641] Writing DB stats nova_cell1:SELECT=100 {{(pid=125641) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 00:43:40.777820 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-71867d7d-cddb-460e-85ed-0245ad1b4cc4 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:43:40.778472 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-71867d7d-cddb-460e-85ed-0245ad1b4cc4 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:43:40.790540 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-71867d7d-cddb-460e-85ed-0245ad1b4cc4 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] 10.222.0.26 "GET /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2" status: 200 len: 1808 microversion: 2.20 time: 0.710648 Oct 19 00:43:40.798134 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 278/393] 10.222.0.26 () {60 vars in 1277 bytes} [Wed Oct 19 00:43:40 2022] GET /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2 => generated 1808 bytes in 720 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 19 00:43:41.826734 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-a3c7a30f-cd40-444d-889f-4cfb4c7a9279 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:43:41.828497 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-a3c7a30f-cd40-444d-889f-4cfb4c7a9279 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] [instance: b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:43:41.844173 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-a3c7a30f-cd40-444d-889f-4cfb4c7a9279 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:43:41.847594 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-a3c7a30f-cd40-444d-889f-4cfb4c7a9279 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:43:41.848737 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-a3c7a30f-cd40-444d-889f-4cfb4c7a9279 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:43:41.851102 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-a3c7a30f-cd40-444d-889f-4cfb4c7a9279 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:43:41.851846 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-a3c7a30f-cd40-444d-889f-4cfb4c7a9279 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:43:41.856702 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-a3c7a30f-cd40-444d-889f-4cfb4c7a9279 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:43:42.322388 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-a3c7a30f-cd40-444d-889f-4cfb4c7a9279 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:43:42.341320 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-a3c7a30f-cd40-444d-889f-4cfb4c7a9279 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:43:42.343793 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-a3c7a30f-cd40-444d-889f-4cfb4c7a9279 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:43:42.345899 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-a3c7a30f-cd40-444d-889f-4cfb4c7a9279 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] 10.222.0.26 "GET /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2" status: 200 len: 1808 microversion: 2.20 time: 0.524142 Oct 19 00:43:42.388306 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 279/394] 10.222.0.26 () {60 vars in 1277 bytes} [Wed Oct 19 00:43:41 2022] GET /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2 => generated 1808 bytes in 568 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 19 00:43:43.414603 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-145452fe-6e67-4138-8a85-e72ecd258704 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:43:43.415900 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-145452fe-6e67-4138-8a85-e72ecd258704 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] [instance: b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:43:43.427554 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-145452fe-6e67-4138-8a85-e72ecd258704 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:43:43.429057 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-145452fe-6e67-4138-8a85-e72ecd258704 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:43:43.430881 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-145452fe-6e67-4138-8a85-e72ecd258704 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:43:43.432458 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-145452fe-6e67-4138-8a85-e72ecd258704 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:43:43.434778 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-145452fe-6e67-4138-8a85-e72ecd258704 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:43:43.435413 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-145452fe-6e67-4138-8a85-e72ecd258704 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:43:43.843421 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-145452fe-6e67-4138-8a85-e72ecd258704 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:43:43.853958 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-145452fe-6e67-4138-8a85-e72ecd258704 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:43:43.855615 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-145452fe-6e67-4138-8a85-e72ecd258704 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:43:43.857206 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-145452fe-6e67-4138-8a85-e72ecd258704 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] 10.222.0.26 "GET /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2" status: 200 len: 1808 microversion: 2.20 time: 0.456064 Oct 19 00:43:43.857823 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 280/395] 10.222.0.26 () {60 vars in 1277 bytes} [Wed Oct 19 00:43:43 2022] GET /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2 => generated 1808 bytes in 459 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 19 00:43:44.894588 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-ce05a04d-7b1d-4a78-9d3d-ffc82b4e2c2a tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:43:44.897048 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-ce05a04d-7b1d-4a78-9d3d-ffc82b4e2c2a tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] [instance: b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:43:44.908129 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ce05a04d-7b1d-4a78-9d3d-ffc82b4e2c2a tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:43:44.908932 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ce05a04d-7b1d-4a78-9d3d-ffc82b4e2c2a tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:43:44.909898 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ce05a04d-7b1d-4a78-9d3d-ffc82b4e2c2a tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:43:44.911444 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ce05a04d-7b1d-4a78-9d3d-ffc82b4e2c2a tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:43:44.912475 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ce05a04d-7b1d-4a78-9d3d-ffc82b4e2c2a tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:43:44.913436 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ce05a04d-7b1d-4a78-9d3d-ffc82b4e2c2a tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:43:45.205083 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-ce05a04d-7b1d-4a78-9d3d-ffc82b4e2c2a tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:43:45.225059 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-ce05a04d-7b1d-4a78-9d3d-ffc82b4e2c2a tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:43:45.226684 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-ce05a04d-7b1d-4a78-9d3d-ffc82b4e2c2a tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:43:45.229131 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-ce05a04d-7b1d-4a78-9d3d-ffc82b4e2c2a tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] 10.222.0.26 "GET /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2" status: 200 len: 1808 microversion: 2.20 time: 0.342414 Oct 19 00:43:45.231114 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 281/396] 10.222.0.26 () {60 vars in 1277 bytes} [Wed Oct 19 00:43:44 2022] GET /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2 => generated 1808 bytes in 345 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 19 00:43:46.285172 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-9e59d8a4-c045-4622-9be2-8110e341db4d tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:43:46.286306 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-9e59d8a4-c045-4622-9be2-8110e341db4d tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] [instance: b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:43:46.294457 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-9e59d8a4-c045-4622-9be2-8110e341db4d tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:43:46.296351 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-9e59d8a4-c045-4622-9be2-8110e341db4d tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:43:46.296351 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-9e59d8a4-c045-4622-9be2-8110e341db4d tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:43:46.296351 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-9e59d8a4-c045-4622-9be2-8110e341db4d tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:43:46.299207 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-9e59d8a4-c045-4622-9be2-8110e341db4d tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:43:46.299207 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-9e59d8a4-c045-4622-9be2-8110e341db4d tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:43:46.489516 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-9e59d8a4-c045-4622-9be2-8110e341db4d tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:43:46.500893 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-9e59d8a4-c045-4622-9be2-8110e341db4d tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:43:46.502567 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-9e59d8a4-c045-4622-9be2-8110e341db4d tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:43:46.503975 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-9e59d8a4-c045-4622-9be2-8110e341db4d tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] 10.222.0.26 "GET /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2" status: 200 len: 1808 microversion: 2.20 time: 0.225992 Oct 19 00:43:46.504884 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 282/397] 10.222.0.26 () {60 vars in 1277 bytes} [Wed Oct 19 00:43:46 2022] GET /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2 => generated 1808 bytes in 228 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 19 00:43:47.534034 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-40b7250b-2e33-447e-af40-515c3ae4ae52 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:43:47.535252 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-40b7250b-2e33-447e-af40-515c3ae4ae52 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] [instance: b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:43:47.559965 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-40b7250b-2e33-447e-af40-515c3ae4ae52 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:43:47.560902 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-40b7250b-2e33-447e-af40-515c3ae4ae52 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:43:47.563257 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-40b7250b-2e33-447e-af40-515c3ae4ae52 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:43:47.564181 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-40b7250b-2e33-447e-af40-515c3ae4ae52 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:43:47.564749 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-40b7250b-2e33-447e-af40-515c3ae4ae52 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:43:47.565432 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-40b7250b-2e33-447e-af40-515c3ae4ae52 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:43:47.847462 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-40b7250b-2e33-447e-af40-515c3ae4ae52 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:43:47.880220 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-40b7250b-2e33-447e-af40-515c3ae4ae52 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:43:47.884351 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-40b7250b-2e33-447e-af40-515c3ae4ae52 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:43:47.890328 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-40b7250b-2e33-447e-af40-515c3ae4ae52 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] 10.222.0.26 "GET /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2" status: 200 len: 1808 microversion: 2.20 time: 0.364264 Oct 19 00:43:47.891559 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 283/398] 10.222.0.26 () {60 vars in 1277 bytes} [Wed Oct 19 00:43:47 2022] GET /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2 => generated 1808 bytes in 367 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 19 00:43:48.918019 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-d5a7da23-33d2-40e6-a7c3-0f69b798289b tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:43:48.918865 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-d5a7da23-33d2-40e6-a7c3-0f69b798289b tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] [instance: b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:43:48.952593 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d5a7da23-33d2-40e6-a7c3-0f69b798289b tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:43:48.953326 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d5a7da23-33d2-40e6-a7c3-0f69b798289b tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:43:48.953961 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d5a7da23-33d2-40e6-a7c3-0f69b798289b tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:43:48.955119 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d5a7da23-33d2-40e6-a7c3-0f69b798289b tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:43:48.958526 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d5a7da23-33d2-40e6-a7c3-0f69b798289b tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:43:48.959312 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d5a7da23-33d2-40e6-a7c3-0f69b798289b tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:43:49.203741 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-d5a7da23-33d2-40e6-a7c3-0f69b798289b tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:43:49.237536 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-d5a7da23-33d2-40e6-a7c3-0f69b798289b tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:43:49.243556 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-d5a7da23-33d2-40e6-a7c3-0f69b798289b tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:43:49.245583 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-d5a7da23-33d2-40e6-a7c3-0f69b798289b tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] 10.222.0.26 "GET /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2" status: 200 len: 1808 microversion: 2.20 time: 0.332758 Oct 19 00:43:49.246515 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 284/399] 10.222.0.26 () {60 vars in 1277 bytes} [Wed Oct 19 00:43:48 2022] GET /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2 => generated 1808 bytes in 335 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 19 00:43:50.295430 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-05c50fe2-05fc-402e-9bd6-2ff84c22c81a tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:43:50.296644 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-05c50fe2-05fc-402e-9bd6-2ff84c22c81a tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] [instance: b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:43:50.338198 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-05c50fe2-05fc-402e-9bd6-2ff84c22c81a tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:43:50.343391 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-05c50fe2-05fc-402e-9bd6-2ff84c22c81a tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:43:50.344512 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-05c50fe2-05fc-402e-9bd6-2ff84c22c81a tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:43:50.346117 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-05c50fe2-05fc-402e-9bd6-2ff84c22c81a tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:43:50.349996 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-05c50fe2-05fc-402e-9bd6-2ff84c22c81a tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:43:50.355064 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-05c50fe2-05fc-402e-9bd6-2ff84c22c81a tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:43:51.131354 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-05c50fe2-05fc-402e-9bd6-2ff84c22c81a tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:43:51.148477 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-05c50fe2-05fc-402e-9bd6-2ff84c22c81a tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:43:51.151936 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-05c50fe2-05fc-402e-9bd6-2ff84c22c81a tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:43:51.156325 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-05c50fe2-05fc-402e-9bd6-2ff84c22c81a tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] 10.222.0.26 "GET /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2" status: 200 len: 1808 microversion: 2.20 time: 0.869851 Oct 19 00:43:51.158586 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 285/400] 10.222.0.26 () {60 vars in 1277 bytes} [Wed Oct 19 00:43:50 2022] GET /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2 => generated 1808 bytes in 873 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 19 00:43:52.207144 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-df8248ed-63f0-403e-ab97-d1bc25f693c6 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:43:52.208274 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-df8248ed-63f0-403e-ab97-d1bc25f693c6 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] [instance: b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:43:52.220370 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-df8248ed-63f0-403e-ab97-d1bc25f693c6 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:43:52.221409 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-df8248ed-63f0-403e-ab97-d1bc25f693c6 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:43:52.222469 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-df8248ed-63f0-403e-ab97-d1bc25f693c6 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:43:52.223800 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-df8248ed-63f0-403e-ab97-d1bc25f693c6 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:43:52.224628 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-df8248ed-63f0-403e-ab97-d1bc25f693c6 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:43:52.225428 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-df8248ed-63f0-403e-ab97-d1bc25f693c6 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:43:52.481732 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-df8248ed-63f0-403e-ab97-d1bc25f693c6 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:43:52.499874 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-df8248ed-63f0-403e-ab97-d1bc25f693c6 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:43:52.502898 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-df8248ed-63f0-403e-ab97-d1bc25f693c6 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:43:52.505528 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-df8248ed-63f0-403e-ab97-d1bc25f693c6 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] 10.222.0.26 "GET /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2" status: 200 len: 1808 microversion: 2.20 time: 0.303119 Oct 19 00:43:52.506870 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 286/401] 10.222.0.26 () {60 vars in 1277 bytes} [Wed Oct 19 00:43:52 2022] GET /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2 => generated 1808 bytes in 305 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 19 00:43:53.553117 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-4be72fed-6b2e-4350-95be-ebc4b84bcc6e tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:43:53.559029 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-4be72fed-6b2e-4350-95be-ebc4b84bcc6e tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] [instance: b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:43:53.571637 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-4be72fed-6b2e-4350-95be-ebc4b84bcc6e tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:43:53.572535 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-4be72fed-6b2e-4350-95be-ebc4b84bcc6e tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:43:53.573134 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-4be72fed-6b2e-4350-95be-ebc4b84bcc6e tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:43:53.574800 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-4be72fed-6b2e-4350-95be-ebc4b84bcc6e tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:43:53.575593 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-4be72fed-6b2e-4350-95be-ebc4b84bcc6e tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:43:53.576215 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-4be72fed-6b2e-4350-95be-ebc4b84bcc6e tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:43:53.768853 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-4be72fed-6b2e-4350-95be-ebc4b84bcc6e tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:43:53.779818 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-4be72fed-6b2e-4350-95be-ebc4b84bcc6e tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:43:53.781914 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-4be72fed-6b2e-4350-95be-ebc4b84bcc6e tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:43:53.783293 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-4be72fed-6b2e-4350-95be-ebc4b84bcc6e tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] 10.222.0.26 "GET /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2" status: 200 len: 1808 microversion: 2.20 time: 0.241121 Oct 19 00:43:53.783943 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 287/402] 10.222.0.26 () {60 vars in 1277 bytes} [Wed Oct 19 00:43:53 2022] GET /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2 => generated 1808 bytes in 243 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 19 00:43:54.822014 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-9a33823e-6e80-4432-a075-cc84b3d952bf tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:43:54.823199 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-9a33823e-6e80-4432-a075-cc84b3d952bf tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] [instance: b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:43:54.825014 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG dbcounter [-] [125641] Writing DB stats nova_api:SELECT=78,nova_api:UPDATE=1,nova_api:DELETE=1 {{(pid=125641) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 00:43:54.831615 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-9a33823e-6e80-4432-a075-cc84b3d952bf tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:43:54.831922 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-9a33823e-6e80-4432-a075-cc84b3d952bf tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:43:54.832493 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-9a33823e-6e80-4432-a075-cc84b3d952bf tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:43:54.833367 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-9a33823e-6e80-4432-a075-cc84b3d952bf tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:43:54.833900 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-9a33823e-6e80-4432-a075-cc84b3d952bf tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:43:54.834405 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-9a33823e-6e80-4432-a075-cc84b3d952bf tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:43:55.023385 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-9a33823e-6e80-4432-a075-cc84b3d952bf tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:43:55.033412 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-9a33823e-6e80-4432-a075-cc84b3d952bf tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:43:55.034939 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-9a33823e-6e80-4432-a075-cc84b3d952bf tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:43:55.036768 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-9a33823e-6e80-4432-a075-cc84b3d952bf tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] 10.222.0.26 "GET /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2" status: 200 len: 1808 microversion: 2.20 time: 0.218460 Oct 19 00:43:55.037353 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 288/403] 10.222.0.26 () {60 vars in 1277 bytes} [Wed Oct 19 00:43:54 2022] GET /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2 => generated 1808 bytes in 220 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 19 00:43:56.073244 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-d606328a-c9fc-4c5b-8520-6ac900732d14 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:43:56.074839 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-d606328a-c9fc-4c5b-8520-6ac900732d14 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] [instance: b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:43:56.083852 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d606328a-c9fc-4c5b-8520-6ac900732d14 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:43:56.084553 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d606328a-c9fc-4c5b-8520-6ac900732d14 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:43:56.085108 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d606328a-c9fc-4c5b-8520-6ac900732d14 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:43:56.086040 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d606328a-c9fc-4c5b-8520-6ac900732d14 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:43:56.086668 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d606328a-c9fc-4c5b-8520-6ac900732d14 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:43:56.087222 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d606328a-c9fc-4c5b-8520-6ac900732d14 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:43:56.326618 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-d606328a-c9fc-4c5b-8520-6ac900732d14 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:43:56.346339 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-d606328a-c9fc-4c5b-8520-6ac900732d14 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:43:56.349923 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-d606328a-c9fc-4c5b-8520-6ac900732d14 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:43:56.352347 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-d606328a-c9fc-4c5b-8520-6ac900732d14 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] 10.222.0.26 "GET /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2" status: 200 len: 1808 microversion: 2.20 time: 0.283812 Oct 19 00:43:56.353711 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 289/404] 10.222.0.26 () {60 vars in 1277 bytes} [Wed Oct 19 00:43:56 2022] GET /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2 => generated 1808 bytes in 286 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 19 00:43:56.363633 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-d1baea50-1a98-44b3-bafc-4d6fba1308ee tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "75266677-7423-452c-84c0-e03427505bf7", "device": "/dev/sdb"}} {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 19 00:43:56.369382 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-d1baea50-1a98-44b3-bafc-4d6fba1308ee tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] [instance: 5bdc36c3-1db5-4ff2-808d-1295f314d522] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:43:56.383463 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d1baea50-1a98-44b3-bafc-4d6fba1308ee tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:43:56.384755 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d1baea50-1a98-44b3-bafc-4d6fba1308ee tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:43:56.385558 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d1baea50-1a98-44b3-bafc-4d6fba1308ee tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:43:56.387449 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d1baea50-1a98-44b3-bafc-4d6fba1308ee tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:43:56.389047 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d1baea50-1a98-44b3-bafc-4d6fba1308ee tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:43:56.390112 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d1baea50-1a98-44b3-bafc-4d6fba1308ee tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:43:56.902952 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-d1baea50-1a98-44b3-bafc-4d6fba1308ee tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] 10.222.0.26 "POST /compute/v2.1/servers/5bdc36c3-1db5-4ff2-808d-1295f314d522/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.545669 Oct 19 00:43:56.904070 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 290/405] 10.222.0.26 () {60 vars in 1323 bytes} [Wed Oct 19 00:43:56 2022] POST /compute/v2.1/servers/5bdc36c3-1db5-4ff2-808d-1295f314d522/os-volume_attachments => generated 194 bytes in 549 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 19 00:43:57.409285 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-51daeb8c-b3fe-417b-8b43-dc24652c48e4 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:43:57.410843 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-51daeb8c-b3fe-417b-8b43-dc24652c48e4 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] [instance: b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:43:57.424657 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-51daeb8c-b3fe-417b-8b43-dc24652c48e4 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:43:57.425789 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-51daeb8c-b3fe-417b-8b43-dc24652c48e4 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:43:57.426862 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-51daeb8c-b3fe-417b-8b43-dc24652c48e4 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:43:57.428322 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-51daeb8c-b3fe-417b-8b43-dc24652c48e4 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:43:57.429441 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-51daeb8c-b3fe-417b-8b43-dc24652c48e4 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:43:57.430209 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-51daeb8c-b3fe-417b-8b43-dc24652c48e4 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:43:57.720574 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-51daeb8c-b3fe-417b-8b43-dc24652c48e4 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:43:57.738423 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-51daeb8c-b3fe-417b-8b43-dc24652c48e4 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:43:57.740037 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-51daeb8c-b3fe-417b-8b43-dc24652c48e4 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:43:57.742199 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-51daeb8c-b3fe-417b-8b43-dc24652c48e4 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] 10.222.0.26 "GET /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2" status: 200 len: 1808 microversion: 2.20 time: 0.341018 Oct 19 00:43:57.742433 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 291/406] 10.222.0.26 () {60 vars in 1277 bytes} [Wed Oct 19 00:43:57 2022] GET /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2 => generated 1808 bytes in 343 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 19 00:43:58.773109 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-7c0939fe-f875-4779-b2f6-61c16685da96 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:43:58.774442 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-7c0939fe-f875-4779-b2f6-61c16685da96 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] [instance: b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:43:58.787128 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-7c0939fe-f875-4779-b2f6-61c16685da96 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:43:58.788085 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-7c0939fe-f875-4779-b2f6-61c16685da96 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:43:58.789006 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-7c0939fe-f875-4779-b2f6-61c16685da96 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:43:58.790339 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-7c0939fe-f875-4779-b2f6-61c16685da96 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:43:58.791415 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-7c0939fe-f875-4779-b2f6-61c16685da96 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:43:58.792176 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-7c0939fe-f875-4779-b2f6-61c16685da96 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:43:59.094789 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-7c0939fe-f875-4779-b2f6-61c16685da96 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:43:59.114207 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-7c0939fe-f875-4779-b2f6-61c16685da96 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:43:59.118388 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-7c0939fe-f875-4779-b2f6-61c16685da96 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:43:59.120855 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-7c0939fe-f875-4779-b2f6-61c16685da96 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] 10.222.0.26 "GET /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2" status: 200 len: 1808 microversion: 2.20 time: 0.354707 Oct 19 00:43:59.121921 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 292/407] 10.222.0.26 () {60 vars in 1277 bytes} [Wed Oct 19 00:43:58 2022] GET /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2 => generated 1808 bytes in 357 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 19 00:44:00.163754 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-e00c0811-52da-4c6f-b576-3034986b9842 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:44:00.164836 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-e00c0811-52da-4c6f-b576-3034986b9842 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] [instance: b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:44:00.178100 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-e00c0811-52da-4c6f-b576-3034986b9842 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:00.178740 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-e00c0811-52da-4c6f-b576-3034986b9842 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:00.179274 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-e00c0811-52da-4c6f-b576-3034986b9842 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:00.180087 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-e00c0811-52da-4c6f-b576-3034986b9842 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:00.180625 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-e00c0811-52da-4c6f-b576-3034986b9842 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:00.181125 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-e00c0811-52da-4c6f-b576-3034986b9842 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:00.443831 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-e00c0811-52da-4c6f-b576-3034986b9842 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:44:00.473158 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-e00c0811-52da-4c6f-b576-3034986b9842 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:44:00.476169 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-e00c0811-52da-4c6f-b576-3034986b9842 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:44:00.478796 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-e00c0811-52da-4c6f-b576-3034986b9842 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] 10.222.0.26 "GET /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2" status: 200 len: 1808 microversion: 2.20 time: 0.320518 Oct 19 00:44:00.482970 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 293/408] 10.222.0.26 () {60 vars in 1277 bytes} [Wed Oct 19 00:44:00 2022] GET /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2 => generated 1808 bytes in 326 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 19 00:44:01.513982 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-2f86c57e-c093-46f7-bacc-2a657bdf1558 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:44:01.515091 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-2f86c57e-c093-46f7-bacc-2a657bdf1558 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] [instance: b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:44:01.529165 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-2f86c57e-c093-46f7-bacc-2a657bdf1558 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:01.531159 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-2f86c57e-c093-46f7-bacc-2a657bdf1558 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:01.532625 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-2f86c57e-c093-46f7-bacc-2a657bdf1558 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:01.534423 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-2f86c57e-c093-46f7-bacc-2a657bdf1558 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:01.535849 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-2f86c57e-c093-46f7-bacc-2a657bdf1558 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:01.536876 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-2f86c57e-c093-46f7-bacc-2a657bdf1558 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:01.581894 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG dbcounter [-] [125641] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=125641) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 00:44:01.895484 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-2f86c57e-c093-46f7-bacc-2a657bdf1558 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:44:01.921324 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-2f86c57e-c093-46f7-bacc-2a657bdf1558 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:44:01.929050 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-2f86c57e-c093-46f7-bacc-2a657bdf1558 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:44:01.931877 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-2f86c57e-c093-46f7-bacc-2a657bdf1558 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] 10.222.0.26 "GET /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2" status: 200 len: 1808 microversion: 2.20 time: 0.426181 Oct 19 00:44:01.937242 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 294/409] 10.222.0.26 () {60 vars in 1277 bytes} [Wed Oct 19 00:44:01 2022] GET /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2 => generated 1808 bytes in 433 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 19 00:44:02.987665 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-de56ff22-87b1-4a21-8348-431004ddb830 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:44:02.990032 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-de56ff22-87b1-4a21-8348-431004ddb830 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] [instance: b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:44:03.005760 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-de56ff22-87b1-4a21-8348-431004ddb830 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:03.006645 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-de56ff22-87b1-4a21-8348-431004ddb830 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:03.007713 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-de56ff22-87b1-4a21-8348-431004ddb830 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:03.009051 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-de56ff22-87b1-4a21-8348-431004ddb830 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:03.009912 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-de56ff22-87b1-4a21-8348-431004ddb830 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:03.010706 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-de56ff22-87b1-4a21-8348-431004ddb830 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:03.233011 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-de56ff22-87b1-4a21-8348-431004ddb830 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:44:03.265041 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-de56ff22-87b1-4a21-8348-431004ddb830 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:44:03.269814 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-de56ff22-87b1-4a21-8348-431004ddb830 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:44:03.279011 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-de56ff22-87b1-4a21-8348-431004ddb830 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] 10.222.0.26 "GET /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2" status: 200 len: 1808 microversion: 2.20 time: 0.298058 Oct 19 00:44:03.296633 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 295/410] 10.222.0.26 () {60 vars in 1277 bytes} [Wed Oct 19 00:44:02 2022] GET /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2 => generated 1808 bytes in 319 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 19 00:44:04.334743 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-c4f8cb2d-5f01-4033-867c-dba1cfce40be tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:44:04.337734 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-c4f8cb2d-5f01-4033-867c-dba1cfce40be tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] [instance: b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:44:04.359405 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-c4f8cb2d-5f01-4033-867c-dba1cfce40be tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:04.360440 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-c4f8cb2d-5f01-4033-867c-dba1cfce40be tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:04.361763 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-c4f8cb2d-5f01-4033-867c-dba1cfce40be tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:04.362859 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-c4f8cb2d-5f01-4033-867c-dba1cfce40be tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:04.365146 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-c4f8cb2d-5f01-4033-867c-dba1cfce40be tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:04.366633 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-c4f8cb2d-5f01-4033-867c-dba1cfce40be tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:04.621112 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-c4f8cb2d-5f01-4033-867c-dba1cfce40be tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:44:04.633860 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-c4f8cb2d-5f01-4033-867c-dba1cfce40be tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:44:04.636455 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-c4f8cb2d-5f01-4033-867c-dba1cfce40be tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:44:04.638460 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-c4f8cb2d-5f01-4033-867c-dba1cfce40be tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] 10.222.0.26 "GET /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2" status: 200 len: 1808 microversion: 2.20 time: 0.311126 Oct 19 00:44:04.652908 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 296/411] 10.222.0.26 () {60 vars in 1277 bytes} [Wed Oct 19 00:44:04 2022] GET /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2 => generated 1808 bytes in 327 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 19 00:44:04.672350 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [req-ff853954-ef24-4130-88fe-b42e9c03006b req-2bc715ce-fbb7-41bf-8873-06feabe47df8 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2", "name": "network-vif-plugged", "status": "completed", "tag": "6aea38b4-b7a0-4e8e-82dc-deb19c260092"}]} {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 19 00:44:04.686515 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-ff853954-ef24-4130-88fe-b42e9c03006b req-2bc715ce-fbb7-41bf-8873-06feabe47df8 service nova] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:04.687163 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-ff853954-ef24-4130-88fe-b42e9c03006b req-2bc715ce-fbb7-41bf-8873-06feabe47df8 service nova] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:04.687705 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-ff853954-ef24-4130-88fe-b42e9c03006b req-2bc715ce-fbb7-41bf-8873-06feabe47df8 service nova] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:04.712702 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.compute.server_external_events [req-ff853954-ef24-4130-88fe-b42e9c03006b req-2bc715ce-fbb7-41bf-8873-06feabe47df8 service nova] Creating event network-vif-plugged:6aea38b4-b7a0-4e8e-82dc-deb19c260092 for instance b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2 on hv-cinder-maste Oct 19 00:44:04.724818 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [req-ff853954-ef24-4130-88fe-b42e9c03006b req-2bc715ce-fbb7-41bf-8873-06feabe47df8 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.070303 Oct 19 00:44:04.726790 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 297/412] 10.222.0.26 () {64 vars in 1317 bytes} [Wed Oct 19 00:44:04 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 73 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 19 00:44:04.738590 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-5ec2a278-283f-4169-a693-67d2481afcd3 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:44:04.740224 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-5ec2a278-283f-4169-a693-67d2481afcd3 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] [instance: 25d64f9f-18a6-4b6a-92b1-3e8e7d017ab8] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:44:04.749479 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-5ec2a278-283f-4169-a693-67d2481afcd3 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:04.750288 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-5ec2a278-283f-4169-a693-67d2481afcd3 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:04.750934 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-5ec2a278-283f-4169-a693-67d2481afcd3 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:04.751825 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-5ec2a278-283f-4169-a693-67d2481afcd3 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:04.752445 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-5ec2a278-283f-4169-a693-67d2481afcd3 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:04.754045 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-5ec2a278-283f-4169-a693-67d2481afcd3 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:04.790261 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-5ec2a278-283f-4169-a693-67d2481afcd3 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] 10.222.0.26 "GET /compute/v2.1/servers/25d64f9f-18a6-4b6a-92b1-3e8e7d017ab8/ips" status: 200 len: 149 microversion: 2.1 time: 0.061414 Oct 19 00:44:04.791154 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 298/413] 10.222.0.26 () {58 vars in 1248 bytes} [Wed Oct 19 00:44:04 2022] GET /compute/v2.1/servers/25d64f9f-18a6-4b6a-92b1-3e8e7d017ab8/ips => generated 149 bytes in 64 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 19 00:44:04.835374 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-8f69a043-abb2-4002-baa2-0054068e6e86 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "6ccd7014-9130-42d2-add5-0b6bc74def3f"}} {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 19 00:44:04.847937 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-8f69a043-abb2-4002-baa2-0054068e6e86 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] [instance: 25d64f9f-18a6-4b6a-92b1-3e8e7d017ab8] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:44:04.856755 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-8f69a043-abb2-4002-baa2-0054068e6e86 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:04.858186 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-8f69a043-abb2-4002-baa2-0054068e6e86 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:04.858842 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-8f69a043-abb2-4002-baa2-0054068e6e86 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:04.859774 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-8f69a043-abb2-4002-baa2-0054068e6e86 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:04.860569 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-8f69a043-abb2-4002-baa2-0054068e6e86 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:04.861377 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-8f69a043-abb2-4002-baa2-0054068e6e86 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:05.044126 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.wsgi [None req-8f69a043-abb2-4002-baa2-0054068e6e86 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] HTTP exception thrown: Invalid volume: volume 6ccd7014-9130-42d2-add5-0b6bc74def3f is already attached to instances: dc8b8a6e-eab7-4f45-8f5d-916c52eec398 Oct 19 00:44:05.046891 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-8f69a043-abb2-4002-baa2-0054068e6e86 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Request method failure captured: Oct 19 00:44:05.046891 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: request: POST /compute/v2.1/servers/25d64f9f-18a6-4b6a-92b1-3e8e7d017ab8/os-volume_attachments HTTP/1.1 Oct 19 00:44:05.046891 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Accept: application/json Oct 19 00:44:05.046891 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Accept-Encoding: identity Oct 19 00:44:05.046891 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Connection: close Oct 19 00:44:05.046891 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Content-Length: 74 Oct 19 00:44:05.046891 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Content-Type: application/json Oct 19 00:44:05.046891 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Host: 10.222.0.26 Oct 19 00:44:05.046891 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Openstack-System-Scope: None Oct 19 00:44:05.046891 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: User-Agent: python-urllib3/1.26.12 Oct 19 00:44:05.046891 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Auth-Token: gAAAAABjT0eJXd9ar1txWHw929SSSa4BnzN-f-jxcCDsiqb5Qu6eweguqGUJXghFM8xaTx1ciC3SJFoac87bZHlU5VANYjjzV6ICQChedig2cB1RmNul9h3AYlunCOiAit1Pl73UlpO5HCAxVXGmz0N3G59VsZacZSMYF6RytJzlde2rg7lCpks Oct 19 00:44:05.046891 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Domain-Id: None Oct 19 00:44:05.046891 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Domain-Name: None Oct 19 00:44:05.046891 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Identity-Status: Confirmed Oct 19 00:44:05.046891 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Is-Admin-Project: True Oct 19 00:44:05.046891 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Project-Domain-Id: default Oct 19 00:44:05.046891 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Project-Domain-Name: Default Oct 19 00:44:05.046891 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Project-Id: 0b35c2038f6b4d0a9012b004f1416c00 Oct 19 00:44:05.046891 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Project-Name: tempest-AttachVolumeNegativeTest-1282551484 Oct 19 00:44:05.046891 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Role: member,reader Oct 19 00:44:05.046891 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Roles: member,reader Oct 19 00:44:05.048645 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/0b35c2038f6b4d0a9012b004f1416c00"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/0b35c2038f6b4d0a9012b004f1416c00"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/0b35c2038f6b4d0a9012b004f1416c00"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_0b35c2038f6b4d0a9012b004f1416c00"}]}] Oct 19 00:44:05.048645 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Tenant: tempest-AttachVolumeNegativeTest-1282551484 Oct 19 00:44:05.048645 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Tenant-Id: 0b35c2038f6b4d0a9012b004f1416c00 Oct 19 00:44:05.048645 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1282551484 Oct 19 00:44:05.048645 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User: tempest-AttachVolumeNegativeTest-1282551484-project Oct 19 00:44:05.048645 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User-Domain-Id: default Oct 19 00:44:05.048645 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User-Domain-Name: Default Oct 19 00:44:05.048645 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User-Id: 6f8725c4d05b4dd79b05b43fba15980b Oct 19 00:44:05.048645 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User-Name: tempest-AttachVolumeNegativeTest-1282551484-project Oct 19 00:44:05.048645 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Oct 19 00:44:05.048645 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: {"volumeAttachment": {"volumeId": "6ccd7014-9130-42d2-add5-0b6bc74def3f"}} Oct 19 00:44:05.048645 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: method: > Oct 19 00:44:05.049448 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: exception: Invalid volume: volume 6ccd7014-9130-42d2-add5-0b6bc74def3f is already attached to instances: dc8b8a6e-eab7-4f45-8f5d-916c52eec398 Oct 19 00:44:05.049448 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: action_args: {'server_id': '25d64f9f-18a6-4b6a-92b1-3e8e7d017ab8', 'body': {'volumeAttachment': {'volumeId': '6ccd7014-9130-42d2-add5-0b6bc74def3f'}}} Oct 19 00:44:05.049448 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 19 00:44:05.049448 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 19 00:44:05.049448 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 388, in create Oct 19 00:44:05.049448 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi device = self.compute_api.attach_volume( Oct 19 00:44:05.049448 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 241, in inner Oct 19 00:44:05.049448 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Oct 19 00:44:05.049448 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 167, in inner Oct 19 00:44:05.049448 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Oct 19 00:44:05.049448 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5130, in attach_volume Oct 19 00:44:05.049448 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi self._check_volume_already_attached(context, instance, volume) Oct 19 00:44:05.049448 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 4970, in _check_volume_already_attached Oct 19 00:44:05.049448 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=msg) Oct 19 00:44:05.049448 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: volume 6ccd7014-9130-42d2-add5-0b6bc74def3f is already attached to instances: dc8b8a6e-eab7-4f45-8f5d-916c52eec398 Oct 19 00:44:05.049448 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:44:05.049448 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 19 00:44:05.049448 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:44:05.049448 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 19 00:44:05.050865 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 19 00:44:05.050865 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 19 00:44:05.050865 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 19 00:44:05.050865 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 19 00:44:05.050865 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 19 00:44:05.050865 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 19 00:44:05.050865 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 19 00:44:05.050865 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 19 00:44:05.050865 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 19 00:44:05.050865 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 19 00:44:05.050865 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 19 00:44:05.050865 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 19 00:44:05.050865 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 406, in create Oct 19 00:44:05.050865 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Oct 19 00:44:05.050865 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: volume 6ccd7014-9130-42d2-add5-0b6bc74def3f is already attached to instances: dc8b8a6e-eab7-4f45-8f5d-916c52eec398 Oct 19 00:44:05.050865 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:44:05.050865 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 19 00:44:05.050865 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:44:05.050865 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 19 00:44:05.050865 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 19 00:44:05.052338 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 19 00:44:05.052338 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 19 00:44:05.052338 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 19 00:44:05.052338 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: volume 6ccd7014-9130-42d2-add5-0b6bc74def3f is already attached to instances: dc8b8a6e-eab7-4f45-8f5d-916c52eec398 Oct 19 00:44:05.052338 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:44:05.056119 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-8f69a043-abb2-4002-baa2-0054068e6e86 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Returning 400 to user: Invalid volume: volume 6ccd7014-9130-42d2-add5-0b6bc74def3f is already attached to instances: dc8b8a6e-eab7-4f45-8f5d-916c52eec398 {{(pid=125641) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 19 00:44:05.058396 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-8f69a043-abb2-4002-baa2-0054068e6e86 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] 10.222.0.26 "POST /compute/v2.1/servers/25d64f9f-18a6-4b6a-92b1-3e8e7d017ab8/os-volume_attachments" status: 400 len: 174 microversion: 2.1 time: 0.250620 Oct 19 00:44:05.064515 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 299/414] 10.222.0.26 () {60 vars in 1323 bytes} [Wed Oct 19 00:44:04 2022] POST /compute/v2.1/servers/25d64f9f-18a6-4b6a-92b1-3e8e7d017ab8/os-volume_attachments => generated 174 bytes in 258 msecs (HTTP/1.1 400) 9 headers in 381 bytes (2 switches on core 0) Oct 19 00:44:05.077076 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-b0974a8b-7f19-4373-807a-614e8b0b2157 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:44:05.080008 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-b0974a8b-7f19-4373-807a-614e8b0b2157 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] [instance: 25d64f9f-18a6-4b6a-92b1-3e8e7d017ab8] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:44:05.090997 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-b0974a8b-7f19-4373-807a-614e8b0b2157 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:05.091760 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-b0974a8b-7f19-4373-807a-614e8b0b2157 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:05.093382 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-b0974a8b-7f19-4373-807a-614e8b0b2157 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:05.094534 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-b0974a8b-7f19-4373-807a-614e8b0b2157 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:05.095473 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-b0974a8b-7f19-4373-807a-614e8b0b2157 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:05.096244 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-b0974a8b-7f19-4373-807a-614e8b0b2157 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:05.137015 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-b0974a8b-7f19-4373-807a-614e8b0b2157 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] [instance: 25d64f9f-18a6-4b6a-92b1-3e8e7d017ab8] Going to try to terminate instance {{(pid=125641) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 19 00:44:05.437139 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-b0974a8b-7f19-4373-807a-614e8b0b2157 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] 10.222.0.26 "DELETE /compute/v2.1/servers/25d64f9f-18a6-4b6a-92b1-3e8e7d017ab8" status: 204 len: 0 microversion: 2.1 time: 0.369528 Oct 19 00:44:05.440004 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 300/415] 10.222.0.26 () {58 vars in 1239 bytes} [Wed Oct 19 00:44:05 2022] DELETE /compute/v2.1/servers/25d64f9f-18a6-4b6a-92b1-3e8e7d017ab8 => generated 0 bytes in 374 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 19 00:44:05.454735 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-e1282fb0-89b8-4753-bdf1-d28756daf48a tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:44:05.455829 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-e1282fb0-89b8-4753-bdf1-d28756daf48a tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] [instance: 25d64f9f-18a6-4b6a-92b1-3e8e7d017ab8] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:44:05.468262 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-e1282fb0-89b8-4753-bdf1-d28756daf48a tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:05.469336 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-e1282fb0-89b8-4753-bdf1-d28756daf48a tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:05.470339 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-e1282fb0-89b8-4753-bdf1-d28756daf48a tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:05.472675 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-e1282fb0-89b8-4753-bdf1-d28756daf48a tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:05.474120 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-e1282fb0-89b8-4753-bdf1-d28756daf48a tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:05.475075 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-e1282fb0-89b8-4753-bdf1-d28756daf48a tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:05.802132 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-e1282fb0-89b8-4753-bdf1-d28756daf48a tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f8725c4d05b4dd79b05b43fba15980b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0b35c2038f6b4d0a9012b004f1416c00', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:44:05.885084 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-e1282fb0-89b8-4753-bdf1-d28756daf48a tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] 10.222.0.26 "GET /compute/v2.1/servers/25d64f9f-18a6-4b6a-92b1-3e8e7d017ab8" status: 200 len: 1681 microversion: 2.1 time: 0.438592 Oct 19 00:44:05.887817 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 301/416] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:44:05 2022] GET /compute/v2.1/servers/25d64f9f-18a6-4b6a-92b1-3e8e7d017ab8 => generated 1681 bytes in 443 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:44:05.910567 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-45b927cf-e30c-48a6-92f1-68ad52d69904 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:44:05.911515 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-45b927cf-e30c-48a6-92f1-68ad52d69904 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] [instance: b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:44:05.928943 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-45b927cf-e30c-48a6-92f1-68ad52d69904 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:05.930518 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-45b927cf-e30c-48a6-92f1-68ad52d69904 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:05.933072 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-45b927cf-e30c-48a6-92f1-68ad52d69904 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:05.934045 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-45b927cf-e30c-48a6-92f1-68ad52d69904 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:05.934706 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-45b927cf-e30c-48a6-92f1-68ad52d69904 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:05.935803 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-45b927cf-e30c-48a6-92f1-68ad52d69904 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:06.269094 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-45b927cf-e30c-48a6-92f1-68ad52d69904 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:44:06.291666 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-45b927cf-e30c-48a6-92f1-68ad52d69904 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:44:06.296190 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-45b927cf-e30c-48a6-92f1-68ad52d69904 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:44:06.298251 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-45b927cf-e30c-48a6-92f1-68ad52d69904 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] 10.222.0.26 "GET /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2" status: 200 len: 1808 microversion: 2.20 time: 0.406498 Oct 19 00:44:06.299267 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 302/417] 10.222.0.26 () {60 vars in 1277 bytes} [Wed Oct 19 00:44:05 2022] GET /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2 => generated 1808 bytes in 409 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 19 00:44:06.932413 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-3347ebbc-4ff0-44ea-9868-925a69944c7e tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:44:06.941922 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-3347ebbc-4ff0-44ea-9868-925a69944c7e tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] [instance: 25d64f9f-18a6-4b6a-92b1-3e8e7d017ab8] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:44:06.966357 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-3347ebbc-4ff0-44ea-9868-925a69944c7e tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:06.967322 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-3347ebbc-4ff0-44ea-9868-925a69944c7e tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:06.970543 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-3347ebbc-4ff0-44ea-9868-925a69944c7e tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:06.971984 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-3347ebbc-4ff0-44ea-9868-925a69944c7e tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:06.972835 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-3347ebbc-4ff0-44ea-9868-925a69944c7e tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:06.973589 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-3347ebbc-4ff0-44ea-9868-925a69944c7e tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:07.259892 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-3347ebbc-4ff0-44ea-9868-925a69944c7e tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f8725c4d05b4dd79b05b43fba15980b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0b35c2038f6b4d0a9012b004f1416c00', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:44:07.291164 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-3347ebbc-4ff0-44ea-9868-925a69944c7e tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] 10.222.0.26 "GET /compute/v2.1/servers/25d64f9f-18a6-4b6a-92b1-3e8e7d017ab8" status: 200 len: 1681 microversion: 2.1 time: 0.365942 Oct 19 00:44:07.293449 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 303/418] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:44:06 2022] GET /compute/v2.1/servers/25d64f9f-18a6-4b6a-92b1-3e8e7d017ab8 => generated 1681 bytes in 370 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 19 00:44:07.354637 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-d69d5685-7d7f-4dc1-a53a-c24a91d59c18 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:44:07.355702 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-d69d5685-7d7f-4dc1-a53a-c24a91d59c18 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] [instance: b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:44:07.366812 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d69d5685-7d7f-4dc1-a53a-c24a91d59c18 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:07.367854 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d69d5685-7d7f-4dc1-a53a-c24a91d59c18 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:07.368765 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d69d5685-7d7f-4dc1-a53a-c24a91d59c18 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:07.370991 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d69d5685-7d7f-4dc1-a53a-c24a91d59c18 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:07.372045 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d69d5685-7d7f-4dc1-a53a-c24a91d59c18 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:07.372937 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d69d5685-7d7f-4dc1-a53a-c24a91d59c18 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:07.404791 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Oct 19 00:44:07.406099 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: Wed Oct 19 00:44:07 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers (ip 10.222.0.26) !!! Oct 19 00:44:07.596433 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-d69d5685-7d7f-4dc1-a53a-c24a91d59c18 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:44:07.609786 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-d69d5685-7d7f-4dc1-a53a-c24a91d59c18 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:44:07.611795 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-d69d5685-7d7f-4dc1-a53a-c24a91d59c18 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:44:07.613512 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-d69d5685-7d7f-4dc1-a53a-c24a91d59c18 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] 10.222.0.26 "GET /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2" status: 200 len: 1808 microversion: 2.20 time: 0.266618 Oct 19 00:44:07.614274 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 304/419] 10.222.0.26 () {60 vars in 1277 bytes} [Wed Oct 19 00:44:07 2022] GET /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2 => generated 1808 bytes in 269 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 19 00:44:08.341988 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-b36b9dd7-fa54-4cfd-b4cc-bf37c9839dca tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:44:08.347043 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-b36b9dd7-fa54-4cfd-b4cc-bf37c9839dca tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] [instance: 25d64f9f-18a6-4b6a-92b1-3e8e7d017ab8] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:44:08.361829 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-b36b9dd7-fa54-4cfd-b4cc-bf37c9839dca tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:08.364349 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-b36b9dd7-fa54-4cfd-b4cc-bf37c9839dca tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:08.366848 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-b36b9dd7-fa54-4cfd-b4cc-bf37c9839dca tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:08.369697 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-b36b9dd7-fa54-4cfd-b4cc-bf37c9839dca tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:08.371658 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-b36b9dd7-fa54-4cfd-b4cc-bf37c9839dca tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:08.381759 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-b36b9dd7-fa54-4cfd-b4cc-bf37c9839dca tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:08.676374 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-b36b9dd7-fa54-4cfd-b4cc-bf37c9839dca tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f8725c4d05b4dd79b05b43fba15980b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0b35c2038f6b4d0a9012b004f1416c00', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:44:08.693387 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-b36b9dd7-fa54-4cfd-b4cc-bf37c9839dca tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] 10.222.0.26 "GET /compute/v2.1/servers/25d64f9f-18a6-4b6a-92b1-3e8e7d017ab8" status: 200 len: 1681 microversion: 2.1 time: 0.358604 Oct 19 00:44:08.694902 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 305/420] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:44:08 2022] GET /compute/v2.1/servers/25d64f9f-18a6-4b6a-92b1-3e8e7d017ab8 => generated 1681 bytes in 361 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:44:08.704828 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-d5c91377-0d89-47e0-a15f-113939611ce4 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:44:08.706318 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-d5c91377-0d89-47e0-a15f-113939611ce4 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] [instance: b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:44:08.721194 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d5c91377-0d89-47e0-a15f-113939611ce4 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:08.723351 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d5c91377-0d89-47e0-a15f-113939611ce4 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:08.724465 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d5c91377-0d89-47e0-a15f-113939611ce4 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:08.726015 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d5c91377-0d89-47e0-a15f-113939611ce4 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:08.727637 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d5c91377-0d89-47e0-a15f-113939611ce4 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:08.728817 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d5c91377-0d89-47e0-a15f-113939611ce4 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:09.016146 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-d5c91377-0d89-47e0-a15f-113939611ce4 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:44:09.026896 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-d5c91377-0d89-47e0-a15f-113939611ce4 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:44:09.028811 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-d5c91377-0d89-47e0-a15f-113939611ce4 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:44:09.030279 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-d5c91377-0d89-47e0-a15f-113939611ce4 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] 10.222.0.26 "GET /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2" status: 200 len: 1808 microversion: 2.20 time: 0.333957 Oct 19 00:44:09.047024 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 306/421] 10.222.0.26 () {60 vars in 1277 bytes} [Wed Oct 19 00:44:08 2022] GET /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2 => generated 1808 bytes in 351 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 19 00:44:09.971275 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-abb68937-bb17-4945-8ebf-560bfada77df tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Action: 'create', calling method: .version_select at 0x7f71c3ec0940>, body: {"keypair": {"name": "tempest-TestVolumeBootPattern-1204843348", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBJnKFjpOepyROSSqIvwbkTVk7xPpRlIi2uavpj5h+qmRfDgD2QoFvuAA+V589Dv4jX35hAfY+YKGfWkh+vh5v9/hgAzWPUo+uEhWwCEom9jHGYjebUawHC3WTHW/EHvhjg=="}} {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 19 00:44:09.999242 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.quota [None req-abb68937-bb17-4945-8ebf-560bfada77df tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Getting quotas for project 8684504b0eff415b9063cebae21ca667. Resources: {'key_pairs'} {{(pid=125641) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 19 00:44:10.005001 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.quota [None req-abb68937-bb17-4945-8ebf-560bfada77df tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Getting quotas for user f738e67d8c2340028da1254b60d7810e and project 8684504b0eff415b9063cebae21ca667. Resources: {'key_pairs'} {{(pid=125641) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 19 00:44:10.087138 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-abb68937-bb17-4945-8ebf-560bfada77df tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] 10.222.0.26 "POST /compute/v2.1/os-keypairs" status: 200 len: 400 microversion: 2.1 time: 0.427158 Oct 19 00:44:10.090062 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 307/422] 10.222.0.26 () {60 vars in 1159 bytes} [Wed Oct 19 00:44:09 2022] POST /compute/v2.1/os-keypairs => generated 400 bytes in 432 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 19 00:44:10.097510 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-9411d402-7c97-49cd-b38e-3e802da002fe tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:44:10.099206 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-9411d402-7c97-49cd-b38e-3e802da002fe tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] [instance: 25d64f9f-18a6-4b6a-92b1-3e8e7d017ab8] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:44:10.109009 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-9411d402-7c97-49cd-b38e-3e802da002fe tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:10.109602 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-9411d402-7c97-49cd-b38e-3e802da002fe tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:10.110111 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-9411d402-7c97-49cd-b38e-3e802da002fe tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:10.110953 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-9411d402-7c97-49cd-b38e-3e802da002fe tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:10.111510 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-9411d402-7c97-49cd-b38e-3e802da002fe tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:10.112217 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-9411d402-7c97-49cd-b38e-3e802da002fe tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:10.385136 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-9411d402-7c97-49cd-b38e-3e802da002fe tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f8725c4d05b4dd79b05b43fba15980b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0b35c2038f6b4d0a9012b004f1416c00', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:44:10.403513 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-9411d402-7c97-49cd-b38e-3e802da002fe tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] 10.222.0.26 "GET /compute/v2.1/servers/25d64f9f-18a6-4b6a-92b1-3e8e7d017ab8" status: 200 len: 1681 microversion: 2.1 time: 0.311475 Oct 19 00:44:10.414549 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 308/423] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:44:10 2022] GET /compute/v2.1/servers/25d64f9f-18a6-4b6a-92b1-3e8e7d017ab8 => generated 1681 bytes in 314 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:44:10.423282 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-c5569b2f-8cc7-4f76-8bf7-e8c98bcef729 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:44:10.424879 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-c5569b2f-8cc7-4f76-8bf7-e8c98bcef729 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] [instance: b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:44:10.448021 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-c5569b2f-8cc7-4f76-8bf7-e8c98bcef729 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:10.448919 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-c5569b2f-8cc7-4f76-8bf7-e8c98bcef729 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:10.449613 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-c5569b2f-8cc7-4f76-8bf7-e8c98bcef729 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:10.451021 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-c5569b2f-8cc7-4f76-8bf7-e8c98bcef729 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:10.452011 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-c5569b2f-8cc7-4f76-8bf7-e8c98bcef729 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:10.452940 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-c5569b2f-8cc7-4f76-8bf7-e8c98bcef729 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:10.725185 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-c5569b2f-8cc7-4f76-8bf7-e8c98bcef729 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:44:10.756389 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-c5569b2f-8cc7-4f76-8bf7-e8c98bcef729 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:44:10.759555 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-c5569b2f-8cc7-4f76-8bf7-e8c98bcef729 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:44:10.764599 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-c5569b2f-8cc7-4f76-8bf7-e8c98bcef729 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] 10.222.0.26 "GET /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2" status: 200 len: 1808 microversion: 2.20 time: 0.355228 Oct 19 00:44:10.765686 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 309/424] 10.222.0.26 () {60 vars in 1277 bytes} [Wed Oct 19 00:44:10 2022] GET /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2 => generated 1808 bytes in 360 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 19 00:44:11.452543 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-61574b02-ae9d-4494-bf39-7a518c1f046d tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:44:11.458020 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-61574b02-ae9d-4494-bf39-7a518c1f046d tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] [instance: 25d64f9f-18a6-4b6a-92b1-3e8e7d017ab8] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:44:11.470663 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-61574b02-ae9d-4494-bf39-7a518c1f046d tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:11.472052 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-61574b02-ae9d-4494-bf39-7a518c1f046d tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:11.473238 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-61574b02-ae9d-4494-bf39-7a518c1f046d tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:11.474965 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-61574b02-ae9d-4494-bf39-7a518c1f046d tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:11.476319 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-61574b02-ae9d-4494-bf39-7a518c1f046d tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:11.476888 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-61574b02-ae9d-4494-bf39-7a518c1f046d tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:11.658899 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-61574b02-ae9d-4494-bf39-7a518c1f046d tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f8725c4d05b4dd79b05b43fba15980b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0b35c2038f6b4d0a9012b004f1416c00', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:44:11.674030 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-61574b02-ae9d-4494-bf39-7a518c1f046d tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] 10.222.0.26 "GET /compute/v2.1/servers/25d64f9f-18a6-4b6a-92b1-3e8e7d017ab8" status: 200 len: 1613 microversion: 2.1 time: 0.229043 Oct 19 00:44:11.676306 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 310/425] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:44:11 2022] GET /compute/v2.1/servers/25d64f9f-18a6-4b6a-92b1-3e8e7d017ab8 => generated 1613 bytes in 233 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:44:11.789217 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [req-a7fd4ce3-466a-4299-9e28-580985cf5f07 req-b29161b8-f164-4ecb-a098-99b0a9ba5199 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "25d64f9f-18a6-4b6a-92b1-3e8e7d017ab8", "name": "network-vif-deleted", "tag": "720bfaf8-2ab1-4a23-bccc-396cef466b3a"}]} {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 19 00:44:11.815628 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-a7fd4ce3-466a-4299-9e28-580985cf5f07 req-b29161b8-f164-4ecb-a098-99b0a9ba5199 service nova] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:11.818916 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-a7fd4ce3-466a-4299-9e28-580985cf5f07 req-b29161b8-f164-4ecb-a098-99b0a9ba5199 service nova] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:11.819964 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-a7fd4ce3-466a-4299-9e28-580985cf5f07 req-b29161b8-f164-4ecb-a098-99b0a9ba5199 service nova] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:11.830903 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG dbcounter [-] [125641] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=125641) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 00:44:11.858551 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.compute.server_external_events [req-a7fd4ce3-466a-4299-9e28-580985cf5f07 req-b29161b8-f164-4ecb-a098-99b0a9ba5199 service nova] Creating event network-vif-deleted:720bfaf8-2ab1-4a23-bccc-396cef466b3a for instance 25d64f9f-18a6-4b6a-92b1-3e8e7d017ab8 on hv-cinder-maste Oct 19 00:44:11.867961 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [req-a7fd4ce3-466a-4299-9e28-580985cf5f07 req-b29161b8-f164-4ecb-a098-99b0a9ba5199 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.093659 Oct 19 00:44:11.874566 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 311/426] 10.222.0.26 () {64 vars in 1317 bytes} [Wed Oct 19 00:44:11 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 19 00:44:11.885309 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-ecaa61e0-ebff-4991-8d3a-9c0bb5e005f4 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:44:11.888110 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-ecaa61e0-ebff-4991-8d3a-9c0bb5e005f4 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] [instance: b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:44:11.912285 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ecaa61e0-ebff-4991-8d3a-9c0bb5e005f4 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:11.913494 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ecaa61e0-ebff-4991-8d3a-9c0bb5e005f4 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.011s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:11.914608 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ecaa61e0-ebff-4991-8d3a-9c0bb5e005f4 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:11.916187 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ecaa61e0-ebff-4991-8d3a-9c0bb5e005f4 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:11.917389 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ecaa61e0-ebff-4991-8d3a-9c0bb5e005f4 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:11.918935 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ecaa61e0-ebff-4991-8d3a-9c0bb5e005f4 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:12.266842 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-ecaa61e0-ebff-4991-8d3a-9c0bb5e005f4 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:44:12.289866 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-ecaa61e0-ebff-4991-8d3a-9c0bb5e005f4 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:44:12.291373 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-ecaa61e0-ebff-4991-8d3a-9c0bb5e005f4 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:44:12.299346 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-ecaa61e0-ebff-4991-8d3a-9c0bb5e005f4 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] 10.222.0.26 "GET /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2" status: 200 len: 1808 microversion: 2.20 time: 0.422245 Oct 19 00:44:12.299964 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 312/427] 10.222.0.26 () {60 vars in 1277 bytes} [Wed Oct 19 00:44:11 2022] GET /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2 => generated 1808 bytes in 425 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 19 00:44:12.713546 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-238074eb-b200-4850-a7f8-0311ce9b5b6d tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:44:12.715022 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-238074eb-b200-4850-a7f8-0311ce9b5b6d tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] [instance: 25d64f9f-18a6-4b6a-92b1-3e8e7d017ab8] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:44:12.722417 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-238074eb-b200-4850-a7f8-0311ce9b5b6d tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:12.722996 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-238074eb-b200-4850-a7f8-0311ce9b5b6d tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:12.723396 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-238074eb-b200-4850-a7f8-0311ce9b5b6d tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:12.724324 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-238074eb-b200-4850-a7f8-0311ce9b5b6d tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:12.724786 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-238074eb-b200-4850-a7f8-0311ce9b5b6d tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:12.725216 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-238074eb-b200-4850-a7f8-0311ce9b5b6d tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:12.872627 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-238074eb-b200-4850-a7f8-0311ce9b5b6d tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f8725c4d05b4dd79b05b43fba15980b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0b35c2038f6b4d0a9012b004f1416c00', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:44:12.889762 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-238074eb-b200-4850-a7f8-0311ce9b5b6d tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] 10.222.0.26 "GET /compute/v2.1/servers/25d64f9f-18a6-4b6a-92b1-3e8e7d017ab8" status: 200 len: 1613 microversion: 2.1 time: 0.179807 Oct 19 00:44:12.891255 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 313/428] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:44:12 2022] GET /compute/v2.1/servers/25d64f9f-18a6-4b6a-92b1-3e8e7d017ab8 => generated 1613 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:44:13.346091 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-2267fa8b-6d0a-4b6a-8891-c20763cb9de4 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:44:13.347098 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-2267fa8b-6d0a-4b6a-8891-c20763cb9de4 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] [instance: b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:44:13.357551 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-2267fa8b-6d0a-4b6a-8891-c20763cb9de4 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:13.358253 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-2267fa8b-6d0a-4b6a-8891-c20763cb9de4 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:13.358895 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-2267fa8b-6d0a-4b6a-8891-c20763cb9de4 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:13.359843 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-2267fa8b-6d0a-4b6a-8891-c20763cb9de4 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:13.360637 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-2267fa8b-6d0a-4b6a-8891-c20763cb9de4 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:13.361321 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-2267fa8b-6d0a-4b6a-8891-c20763cb9de4 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:13.769196 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-2267fa8b-6d0a-4b6a-8891-c20763cb9de4 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:44:13.791236 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-2267fa8b-6d0a-4b6a-8891-c20763cb9de4 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:44:13.795623 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-2267fa8b-6d0a-4b6a-8891-c20763cb9de4 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:44:13.798607 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-2267fa8b-6d0a-4b6a-8891-c20763cb9de4 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] 10.222.0.26 "GET /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2" status: 200 len: 1808 microversion: 2.20 time: 0.463504 Oct 19 00:44:13.800917 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 314/429] 10.222.0.26 () {60 vars in 1277 bytes} [Wed Oct 19 00:44:13 2022] GET /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2 => generated 1808 bytes in 468 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 19 00:44:13.918104 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-32ec059b-2bbf-4518-96da-476ac2fe370c tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:44:13.919096 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-32ec059b-2bbf-4518-96da-476ac2fe370c tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] [instance: 25d64f9f-18a6-4b6a-92b1-3e8e7d017ab8] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:44:13.942234 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-32ec059b-2bbf-4518-96da-476ac2fe370c tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:13.944236 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-32ec059b-2bbf-4518-96da-476ac2fe370c tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:13.944976 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-32ec059b-2bbf-4518-96da-476ac2fe370c tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:13.946022 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-32ec059b-2bbf-4518-96da-476ac2fe370c tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:13.946902 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-32ec059b-2bbf-4518-96da-476ac2fe370c tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:13.952499 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-32ec059b-2bbf-4518-96da-476ac2fe370c tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:14.062971 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-32ec059b-2bbf-4518-96da-476ac2fe370c tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:14.063472 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-32ec059b-2bbf-4518-96da-476ac2fe370c tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:14.066211 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-32ec059b-2bbf-4518-96da-476ac2fe370c tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:14.066726 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.objects.instance [None req-32ec059b-2bbf-4518-96da-476ac2fe370c tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lazy-loading 'fault' on Instance uuid 25d64f9f-18a6-4b6a-92b1-3e8e7d017ab8 {{(pid=125641) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 19 00:44:14.178991 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-32ec059b-2bbf-4518-96da-476ac2fe370c tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f8725c4d05b4dd79b05b43fba15980b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0b35c2038f6b4d0a9012b004f1416c00', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:44:14.190171 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-32ec059b-2bbf-4518-96da-476ac2fe370c tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] 10.222.0.26 "GET /compute/v2.1/servers/25d64f9f-18a6-4b6a-92b1-3e8e7d017ab8" status: 200 len: 1331 microversion: 2.1 time: 0.277086 Oct 19 00:44:14.190771 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 315/430] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:44:13 2022] GET /compute/v2.1/servers/25d64f9f-18a6-4b6a-92b1-3e8e7d017ab8 => generated 1331 bytes in 279 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:44:14.843911 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-355e1694-c6e7-49f4-ad52-3b1517d9ac21 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:44:14.845189 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-355e1694-c6e7-49f4-ad52-3b1517d9ac21 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] [instance: b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:44:14.859266 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-355e1694-c6e7-49f4-ad52-3b1517d9ac21 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:14.862999 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-355e1694-c6e7-49f4-ad52-3b1517d9ac21 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:14.863532 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-355e1694-c6e7-49f4-ad52-3b1517d9ac21 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:14.864296 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-355e1694-c6e7-49f4-ad52-3b1517d9ac21 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:14.870429 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-355e1694-c6e7-49f4-ad52-3b1517d9ac21 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:14.870429 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-355e1694-c6e7-49f4-ad52-3b1517d9ac21 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:15.100993 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-355e1694-c6e7-49f4-ad52-3b1517d9ac21 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:44:15.113613 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-355e1694-c6e7-49f4-ad52-3b1517d9ac21 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:44:15.117866 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-355e1694-c6e7-49f4-ad52-3b1517d9ac21 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:44:15.121454 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-355e1694-c6e7-49f4-ad52-3b1517d9ac21 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] 10.222.0.26 "GET /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2" status: 200 len: 1808 microversion: 2.20 time: 0.283335 Oct 19 00:44:15.122969 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 316/431] 10.222.0.26 () {60 vars in 1277 bytes} [Wed Oct 19 00:44:14 2022] GET /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2 => generated 1808 bytes in 287 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 19 00:44:15.217386 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-2a991777-cd7b-42b8-a91f-c36638e48f0c tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:44:15.219160 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-2a991777-cd7b-42b8-a91f-c36638e48f0c tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] [instance: 25d64f9f-18a6-4b6a-92b1-3e8e7d017ab8] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:44:15.234785 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-2a991777-cd7b-42b8-a91f-c36638e48f0c tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:15.235570 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-2a991777-cd7b-42b8-a91f-c36638e48f0c tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:15.236386 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-2a991777-cd7b-42b8-a91f-c36638e48f0c tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:15.237329 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-2a991777-cd7b-42b8-a91f-c36638e48f0c tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:15.237996 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-2a991777-cd7b-42b8-a91f-c36638e48f0c tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:15.238637 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-2a991777-cd7b-42b8-a91f-c36638e48f0c tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:15.316021 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-2a991777-cd7b-42b8-a91f-c36638e48f0c tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:15.316801 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-2a991777-cd7b-42b8-a91f-c36638e48f0c tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:15.317350 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-2a991777-cd7b-42b8-a91f-c36638e48f0c tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:15.317919 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.objects.instance [None req-2a991777-cd7b-42b8-a91f-c36638e48f0c tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lazy-loading 'fault' on Instance uuid 25d64f9f-18a6-4b6a-92b1-3e8e7d017ab8 {{(pid=125641) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 19 00:44:15.390870 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-2a991777-cd7b-42b8-a91f-c36638e48f0c tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f8725c4d05b4dd79b05b43fba15980b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0b35c2038f6b4d0a9012b004f1416c00', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:44:15.403480 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-2a991777-cd7b-42b8-a91f-c36638e48f0c tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] 10.222.0.26 "GET /compute/v2.1/servers/25d64f9f-18a6-4b6a-92b1-3e8e7d017ab8" status: 200 len: 1331 microversion: 2.1 time: 0.192090 Oct 19 00:44:15.404251 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 317/432] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:44:15 2022] GET /compute/v2.1/servers/25d64f9f-18a6-4b6a-92b1-3e8e7d017ab8 => generated 1331 bytes in 194 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 19 00:44:16.169818 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-6be941fe-933b-4940-a81e-7f2fd68eded9 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:44:16.172939 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-6be941fe-933b-4940-a81e-7f2fd68eded9 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] [instance: b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:44:16.184601 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-6be941fe-933b-4940-a81e-7f2fd68eded9 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:16.185271 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-6be941fe-933b-4940-a81e-7f2fd68eded9 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:16.185986 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-6be941fe-933b-4940-a81e-7f2fd68eded9 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:16.186960 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-6be941fe-933b-4940-a81e-7f2fd68eded9 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:16.187618 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-6be941fe-933b-4940-a81e-7f2fd68eded9 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:16.188182 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-6be941fe-933b-4940-a81e-7f2fd68eded9 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:16.413483 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-6be941fe-933b-4940-a81e-7f2fd68eded9 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:44:16.423699 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-6be941fe-933b-4940-a81e-7f2fd68eded9 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:44:16.425047 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-6be941fe-933b-4940-a81e-7f2fd68eded9 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:44:16.426270 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-6be941fe-933b-4940-a81e-7f2fd68eded9 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] 10.222.0.26 "GET /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2" status: 200 len: 1808 microversion: 2.20 time: 0.262288 Oct 19 00:44:16.427081 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 318/433] 10.222.0.26 () {60 vars in 1277 bytes} [Wed Oct 19 00:44:16 2022] GET /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2 => generated 1808 bytes in 264 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 19 00:44:16.444850 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-7e74904e-2241-4698-a4c4-4889db53fb15 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:44:16.445658 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-7e74904e-2241-4698-a4c4-4889db53fb15 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] [instance: 25d64f9f-18a6-4b6a-92b1-3e8e7d017ab8] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:44:16.456180 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-7e74904e-2241-4698-a4c4-4889db53fb15 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:16.456960 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-7e74904e-2241-4698-a4c4-4889db53fb15 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:16.457521 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-7e74904e-2241-4698-a4c4-4889db53fb15 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:16.458836 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-7e74904e-2241-4698-a4c4-4889db53fb15 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:16.459676 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-7e74904e-2241-4698-a4c4-4889db53fb15 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:16.460402 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-7e74904e-2241-4698-a4c4-4889db53fb15 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:16.516594 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-7e74904e-2241-4698-a4c4-4889db53fb15 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:16.517456 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-7e74904e-2241-4698-a4c4-4889db53fb15 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:16.518232 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-7e74904e-2241-4698-a4c4-4889db53fb15 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:16.519069 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.objects.instance [None req-7e74904e-2241-4698-a4c4-4889db53fb15 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lazy-loading 'fault' on Instance uuid 25d64f9f-18a6-4b6a-92b1-3e8e7d017ab8 {{(pid=125641) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 19 00:44:16.585977 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-7e74904e-2241-4698-a4c4-4889db53fb15 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f8725c4d05b4dd79b05b43fba15980b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0b35c2038f6b4d0a9012b004f1416c00', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:44:16.601437 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-7e74904e-2241-4698-a4c4-4889db53fb15 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] 10.222.0.26 "GET /compute/v2.1/servers/25d64f9f-18a6-4b6a-92b1-3e8e7d017ab8" status: 200 len: 1331 microversion: 2.1 time: 0.160695 Oct 19 00:44:16.602505 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 319/434] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:44:16 2022] GET /compute/v2.1/servers/25d64f9f-18a6-4b6a-92b1-3e8e7d017ab8 => generated 1331 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:44:17.462438 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-b58483b4-2db0-4380-90a4-8e75bbed7a8d tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:44:17.463678 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-b58483b4-2db0-4380-90a4-8e75bbed7a8d tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] [instance: b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:44:17.475277 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-b58483b4-2db0-4380-90a4-8e75bbed7a8d tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:17.476124 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-b58483b4-2db0-4380-90a4-8e75bbed7a8d tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:17.476825 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-b58483b4-2db0-4380-90a4-8e75bbed7a8d tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:17.478021 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-b58483b4-2db0-4380-90a4-8e75bbed7a8d tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:17.478876 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-b58483b4-2db0-4380-90a4-8e75bbed7a8d tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:17.480172 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-b58483b4-2db0-4380-90a4-8e75bbed7a8d tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:17.661769 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-b58483b4-2db0-4380-90a4-8e75bbed7a8d tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:44:17.672489 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-b58483b4-2db0-4380-90a4-8e75bbed7a8d tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:44:17.674062 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-b58483b4-2db0-4380-90a4-8e75bbed7a8d tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:44:17.675536 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-b58483b4-2db0-4380-90a4-8e75bbed7a8d tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] 10.222.0.26 "GET /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2" status: 200 len: 1808 microversion: 2.20 time: 0.220697 Oct 19 00:44:17.676363 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 320/435] 10.222.0.26 () {60 vars in 1277 bytes} [Wed Oct 19 00:44:17 2022] GET /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2 => generated 1808 bytes in 223 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 19 00:44:17.681486 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-e3c52175-8a19-47b9-8aee-c4119f92a986 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:44:17.682215 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-e3c52175-8a19-47b9-8aee-c4119f92a986 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] [instance: 25d64f9f-18a6-4b6a-92b1-3e8e7d017ab8] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:44:17.689178 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-e3c52175-8a19-47b9-8aee-c4119f92a986 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:17.689669 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-e3c52175-8a19-47b9-8aee-c4119f92a986 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:17.690102 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-e3c52175-8a19-47b9-8aee-c4119f92a986 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:17.690836 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-e3c52175-8a19-47b9-8aee-c4119f92a986 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:17.691290 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-e3c52175-8a19-47b9-8aee-c4119f92a986 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:17.691710 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-e3c52175-8a19-47b9-8aee-c4119f92a986 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:17.727080 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-e3c52175-8a19-47b9-8aee-c4119f92a986 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:17.727566 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-e3c52175-8a19-47b9-8aee-c4119f92a986 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:17.728002 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-e3c52175-8a19-47b9-8aee-c4119f92a986 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:17.728449 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.objects.instance [None req-e3c52175-8a19-47b9-8aee-c4119f92a986 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lazy-loading 'fault' on Instance uuid 25d64f9f-18a6-4b6a-92b1-3e8e7d017ab8 {{(pid=125641) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 19 00:44:17.773988 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-e3c52175-8a19-47b9-8aee-c4119f92a986 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f8725c4d05b4dd79b05b43fba15980b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0b35c2038f6b4d0a9012b004f1416c00', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:44:17.788348 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-e3c52175-8a19-47b9-8aee-c4119f92a986 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] 10.222.0.26 "GET /compute/v2.1/servers/25d64f9f-18a6-4b6a-92b1-3e8e7d017ab8" status: 200 len: 1331 microversion: 2.1 time: 0.110649 Oct 19 00:44:17.789267 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 321/436] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:44:17 2022] GET /compute/v2.1/servers/25d64f9f-18a6-4b6a-92b1-3e8e7d017ab8 => generated 1331 bytes in 113 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:44:18.711455 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-64354550-94b2-4c44-a4cb-4b1c80e16ecb tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:44:18.712582 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-64354550-94b2-4c44-a4cb-4b1c80e16ecb tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] [instance: b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:44:18.721697 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-64354550-94b2-4c44-a4cb-4b1c80e16ecb tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:18.722315 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-64354550-94b2-4c44-a4cb-4b1c80e16ecb tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:18.722868 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-64354550-94b2-4c44-a4cb-4b1c80e16ecb tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:18.723892 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-64354550-94b2-4c44-a4cb-4b1c80e16ecb tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:18.724503 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-64354550-94b2-4c44-a4cb-4b1c80e16ecb tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:18.725100 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-64354550-94b2-4c44-a4cb-4b1c80e16ecb tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:18.898925 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-64354550-94b2-4c44-a4cb-4b1c80e16ecb tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:44:18.910819 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-64354550-94b2-4c44-a4cb-4b1c80e16ecb tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:44:18.912835 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-64354550-94b2-4c44-a4cb-4b1c80e16ecb tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:44:18.914147 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-64354550-94b2-4c44-a4cb-4b1c80e16ecb tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] 10.222.0.26 "GET /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2" status: 200 len: 1795 microversion: 2.20 time: 0.209690 Oct 19 00:44:18.915144 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 322/437] 10.222.0.26 () {60 vars in 1277 bytes} [Wed Oct 19 00:44:18 2022] GET /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2 => generated 1795 bytes in 212 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 19 00:44:18.920519 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-14f6c69b-404c-49bf-a3b6-57322a09b0b1 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:44:18.921134 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-14f6c69b-404c-49bf-a3b6-57322a09b0b1 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] [instance: 25d64f9f-18a6-4b6a-92b1-3e8e7d017ab8] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:44:18.927834 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-14f6c69b-404c-49bf-a3b6-57322a09b0b1 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:18.928401 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-14f6c69b-404c-49bf-a3b6-57322a09b0b1 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:18.929148 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-14f6c69b-404c-49bf-a3b6-57322a09b0b1 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:18.929634 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-14f6c69b-404c-49bf-a3b6-57322a09b0b1 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:18.930101 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-14f6c69b-404c-49bf-a3b6-57322a09b0b1 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:18.931249 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-14f6c69b-404c-49bf-a3b6-57322a09b0b1 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:18.954148 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.wsgi [None req-14f6c69b-404c-49bf-a3b6-57322a09b0b1 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] HTTP exception thrown: Instance 25d64f9f-18a6-4b6a-92b1-3e8e7d017ab8 could not be found. Oct 19 00:44:18.956615 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-14f6c69b-404c-49bf-a3b6-57322a09b0b1 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Request method failure captured: Oct 19 00:44:18.956615 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: request: GET /compute/v2.1/servers/25d64f9f-18a6-4b6a-92b1-3e8e7d017ab8 HTTP/1.1 Oct 19 00:44:18.956615 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Accept: application/json Oct 19 00:44:18.956615 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Accept-Encoding: identity Oct 19 00:44:18.956615 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Connection: close Oct 19 00:44:18.956615 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Content-Length: 0 Oct 19 00:44:18.956615 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Content-Type: application/json Oct 19 00:44:18.956615 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Host: 10.222.0.26 Oct 19 00:44:18.956615 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Openstack-System-Scope: None Oct 19 00:44:18.956615 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: User-Agent: python-urllib3/1.26.12 Oct 19 00:44:18.956615 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Auth-Token: gAAAAABjT0eJXd9ar1txWHw929SSSa4BnzN-f-jxcCDsiqb5Qu6eweguqGUJXghFM8xaTx1ciC3SJFoac87bZHlU5VANYjjzV6ICQChedig2cB1RmNul9h3AYlunCOiAit1Pl73UlpO5HCAxVXGmz0N3G59VsZacZSMYF6RytJzlde2rg7lCpks Oct 19 00:44:18.956615 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Domain-Id: None Oct 19 00:44:18.956615 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Domain-Name: None Oct 19 00:44:18.956615 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Identity-Status: Confirmed Oct 19 00:44:18.956615 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Is-Admin-Project: True Oct 19 00:44:18.956615 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Project-Domain-Id: default Oct 19 00:44:18.956615 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Project-Domain-Name: Default Oct 19 00:44:18.956615 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Project-Id: 0b35c2038f6b4d0a9012b004f1416c00 Oct 19 00:44:18.956615 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Project-Name: tempest-AttachVolumeNegativeTest-1282551484 Oct 19 00:44:18.956615 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Role: member,reader Oct 19 00:44:18.956615 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Roles: member,reader Oct 19 00:44:18.958398 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/0b35c2038f6b4d0a9012b004f1416c00"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/0b35c2038f6b4d0a9012b004f1416c00"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/0b35c2038f6b4d0a9012b004f1416c00"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_0b35c2038f6b4d0a9012b004f1416c00"}]}] Oct 19 00:44:18.958398 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Tenant: tempest-AttachVolumeNegativeTest-1282551484 Oct 19 00:44:18.958398 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Tenant-Id: 0b35c2038f6b4d0a9012b004f1416c00 Oct 19 00:44:18.958398 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1282551484 Oct 19 00:44:18.958398 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User: tempest-AttachVolumeNegativeTest-1282551484-project Oct 19 00:44:18.958398 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User-Domain-Id: default Oct 19 00:44:18.958398 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User-Domain-Name: Default Oct 19 00:44:18.958398 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User-Id: 6f8725c4d05b4dd79b05b43fba15980b Oct 19 00:44:18.958398 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User-Name: tempest-AttachVolumeNegativeTest-1282551484-project Oct 19 00:44:18.958398 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: method: > Oct 19 00:44:18.958398 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: exception: Instance 25d64f9f-18a6-4b6a-92b1-3e8e7d017ab8 could not be found. Oct 19 00:44:18.958398 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: action_args: {'id': '25d64f9f-18a6-4b6a-92b1-3e8e7d017ab8'} Oct 19 00:44:18.959808 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 19 00:44:18.959808 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 19 00:44:18.959808 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 19 00:44:18.959808 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 19 00:44:18.959808 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 19 00:44:18.959808 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 19 00:44:18.959808 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 19 00:44:18.959808 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 19 00:44:18.959808 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 19 00:44:18.959808 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 19 00:44:18.959808 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 25d64f9f-18a6-4b6a-92b1-3e8e7d017ab8 could not be found. Oct 19 00:44:18.959808 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:44:18.959808 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 19 00:44:18.959808 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:44:18.959808 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 19 00:44:18.959808 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 19 00:44:18.959808 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 19 00:44:18.959808 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 19 00:44:18.959808 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 19 00:44:18.959808 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 19 00:44:18.961244 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 19 00:44:18.961244 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 19 00:44:18.961244 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 19 00:44:18.961244 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 19 00:44:18.961244 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 19 00:44:18.961244 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 19 00:44:18.961244 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 19 00:44:18.961244 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 25d64f9f-18a6-4b6a-92b1-3e8e7d017ab8 could not be found. Oct 19 00:44:18.961244 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:44:18.961244 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 19 00:44:18.961244 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:44:18.961244 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 19 00:44:18.961244 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 19 00:44:18.961244 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 19 00:44:18.961244 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 19 00:44:18.961244 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 19 00:44:18.961244 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 25d64f9f-18a6-4b6a-92b1-3e8e7d017ab8 could not be found. Oct 19 00:44:18.961244 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:44:18.962681 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-14f6c69b-404c-49bf-a3b6-57322a09b0b1 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Returning 404 to user: Instance 25d64f9f-18a6-4b6a-92b1-3e8e7d017ab8 could not be found. {{(pid=125641) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 19 00:44:18.962681 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-14f6c69b-404c-49bf-a3b6-57322a09b0b1 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] 10.222.0.26 "GET /compute/v2.1/servers/25d64f9f-18a6-4b6a-92b1-3e8e7d017ab8" status: 404 len: 111 microversion: 2.1 time: 0.044672 Oct 19 00:44:18.964155 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 323/438] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:44:18 2022] GET /compute/v2.1/servers/25d64f9f-18a6-4b6a-92b1-3e8e7d017ab8 => generated 111 bytes in 48 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 19 00:44:18.976874 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-6440c862-e826-427c-a0e8-6f7d196e306f tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Calling method '.version_select at 0x7f71c8cff3a0>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:44:19.005272 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-6440c862-e826-427c-a0e8-6f7d196e306f tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] 10.222.0.26 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-356906562" status: 202 len: 0 microversion: 2.1 time: 0.033690 Oct 19 00:44:19.011269 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 324/439] 10.222.0.26 () {58 vars in 1218 bytes} [Wed Oct 19 00:44:18 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-356906562 => generated 0 bytes in 41 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 19 00:44:19.856255 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-70a0a2e3-13be-4a71-8e08-5773f8ca3e43 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:44:19.857107 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-70a0a2e3-13be-4a71-8e08-5773f8ca3e43 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] [instance: dc8b8a6e-eab7-4f45-8f5d-916c52eec398] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:44:19.858903 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG dbcounter [-] [125641] Writing DB stats nova_api:SELECT=97,nova_api:UPDATE=1,nova_api:INSERT=1,nova_api:DELETE=1 {{(pid=125641) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 00:44:19.865832 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-70a0a2e3-13be-4a71-8e08-5773f8ca3e43 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:19.866320 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-70a0a2e3-13be-4a71-8e08-5773f8ca3e43 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:19.866785 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-70a0a2e3-13be-4a71-8e08-5773f8ca3e43 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:19.867499 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-70a0a2e3-13be-4a71-8e08-5773f8ca3e43 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:19.867951 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-70a0a2e3-13be-4a71-8e08-5773f8ca3e43 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:19.868375 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-70a0a2e3-13be-4a71-8e08-5773f8ca3e43 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:20.015912 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG dbcounter [-] [125641] Writing DB stats nova_cell0:SELECT=4 {{(pid=125641) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 00:44:20.065208 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.objects.instance [None req-70a0a2e3-13be-4a71-8e08-5773f8ca3e43 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lazy-loading 'services' on Instance uuid dc8b8a6e-eab7-4f45-8f5d-916c52eec398 {{(pid=125641) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 19 00:44:20.248175 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-70a0a2e3-13be-4a71-8e08-5773f8ca3e43 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] 10.222.0.26 "DELETE /compute/v2.1/servers/dc8b8a6e-eab7-4f45-8f5d-916c52eec398/os-volume_attachments/6ccd7014-9130-42d2-add5-0b6bc74def3f" status: 202 len: 0 microversion: 2.1 time: 0.396907 Oct 19 00:44:20.248842 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 325/440] 10.222.0.26 () {58 vars in 1416 bytes} [Wed Oct 19 00:44:19 2022] DELETE /compute/v2.1/servers/dc8b8a6e-eab7-4f45-8f5d-916c52eec398/os-volume_attachments/6ccd7014-9130-42d2-add5-0b6bc74def3f => generated 0 bytes in 399 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Oct 19 00:44:20.261999 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-42f6b9e6-e392-4e7e-8792-aacd0a30351e tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:44:20.267054 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-42f6b9e6-e392-4e7e-8792-aacd0a30351e tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] [instance: dc8b8a6e-eab7-4f45-8f5d-916c52eec398] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:44:20.273192 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-42f6b9e6-e392-4e7e-8792-aacd0a30351e tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:20.273749 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-42f6b9e6-e392-4e7e-8792-aacd0a30351e tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:20.274227 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-42f6b9e6-e392-4e7e-8792-aacd0a30351e tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:20.275552 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-42f6b9e6-e392-4e7e-8792-aacd0a30351e tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:20.276484 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-42f6b9e6-e392-4e7e-8792-aacd0a30351e tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:20.277032 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-42f6b9e6-e392-4e7e-8792-aacd0a30351e tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:20.315005 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-42f6b9e6-e392-4e7e-8792-aacd0a30351e tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] 10.222.0.26 "GET /compute/v2.1/servers/dc8b8a6e-eab7-4f45-8f5d-916c52eec398/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.060446 Oct 19 00:44:20.315606 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 326/441] 10.222.0.26 () {58 vars in 1302 bytes} [Wed Oct 19 00:44:20 2022] GET /compute/v2.1/servers/dc8b8a6e-eab7-4f45-8f5d-916c52eec398/os-volume_attachments => generated 197 bytes in 62 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 19 00:44:20.321196 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-96086dd4-5789-4c91-9ae7-9cdd20322306 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "b9abcc63-2fc1-4c36-a503-af404a15c9f5"}} {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 19 00:44:20.324586 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-96086dd4-5789-4c91-9ae7-9cdd20322306 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] [instance: f6051269-1d16-42db-8323-67a532dbcf40] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:44:20.335226 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-96086dd4-5789-4c91-9ae7-9cdd20322306 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:20.336344 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-96086dd4-5789-4c91-9ae7-9cdd20322306 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:20.337196 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-96086dd4-5789-4c91-9ae7-9cdd20322306 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:20.339016 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-96086dd4-5789-4c91-9ae7-9cdd20322306 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:20.340071 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-96086dd4-5789-4c91-9ae7-9cdd20322306 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:20.340755 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-96086dd4-5789-4c91-9ae7-9cdd20322306 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:20.951363 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG dbcounter [-] [125641] Writing DB stats nova_cell1:SELECT=98,nova_cell1:INSERT=1,nova_cell1:UPDATE=1 {{(pid=125641) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 00:44:21.103418 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-96086dd4-5789-4c91-9ae7-9cdd20322306 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] 10.222.0.26 "POST /compute/v2.1/servers/f6051269-1d16-42db-8323-67a532dbcf40/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.786513 Oct 19 00:44:21.104040 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 327/442] 10.222.0.26 () {60 vars in 1323 bytes} [Wed Oct 19 00:44:20 2022] POST /compute/v2.1/servers/f6051269-1d16-42db-8323-67a532dbcf40/os-volume_attachments => generated 194 bytes in 788 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 19 00:44:21.336549 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-1bc820c6-ecd1-4f9c-a04b-ca0cbe1c4be1 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:44:21.341596 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-1bc820c6-ecd1-4f9c-a04b-ca0cbe1c4be1 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] [instance: dc8b8a6e-eab7-4f45-8f5d-916c52eec398] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:44:21.353269 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-1bc820c6-ecd1-4f9c-a04b-ca0cbe1c4be1 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:21.354132 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-1bc820c6-ecd1-4f9c-a04b-ca0cbe1c4be1 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:21.354936 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-1bc820c6-ecd1-4f9c-a04b-ca0cbe1c4be1 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:21.356210 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-1bc820c6-ecd1-4f9c-a04b-ca0cbe1c4be1 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:21.357034 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-1bc820c6-ecd1-4f9c-a04b-ca0cbe1c4be1 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:21.357812 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-1bc820c6-ecd1-4f9c-a04b-ca0cbe1c4be1 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:21.419791 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-1bc820c6-ecd1-4f9c-a04b-ca0cbe1c4be1 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] 10.222.0.26 "GET /compute/v2.1/servers/dc8b8a6e-eab7-4f45-8f5d-916c52eec398/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.092310 Oct 19 00:44:21.428794 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 328/443] 10.222.0.26 () {58 vars in 1302 bytes} [Wed Oct 19 00:44:21 2022] GET /compute/v2.1/servers/dc8b8a6e-eab7-4f45-8f5d-916c52eec398/os-volume_attachments => generated 197 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 19 00:44:22.440564 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-0e5d493e-8e53-4a69-9e01-03780b3cb8a5 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:44:22.443615 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-0e5d493e-8e53-4a69-9e01-03780b3cb8a5 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] [instance: dc8b8a6e-eab7-4f45-8f5d-916c52eec398] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:44:22.455965 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-0e5d493e-8e53-4a69-9e01-03780b3cb8a5 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:22.456844 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-0e5d493e-8e53-4a69-9e01-03780b3cb8a5 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:22.457630 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-0e5d493e-8e53-4a69-9e01-03780b3cb8a5 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:22.459062 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-0e5d493e-8e53-4a69-9e01-03780b3cb8a5 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:22.459871 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-0e5d493e-8e53-4a69-9e01-03780b3cb8a5 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:22.461810 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-0e5d493e-8e53-4a69-9e01-03780b3cb8a5 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:22.520187 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-0e5d493e-8e53-4a69-9e01-03780b3cb8a5 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] 10.222.0.26 "GET /compute/v2.1/servers/dc8b8a6e-eab7-4f45-8f5d-916c52eec398/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.086658 Oct 19 00:44:22.520718 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 329/444] 10.222.0.26 () {58 vars in 1302 bytes} [Wed Oct 19 00:44:22 2022] GET /compute/v2.1/servers/dc8b8a6e-eab7-4f45-8f5d-916c52eec398/os-volume_attachments => generated 25 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 356 bytes (2 switches on core 0) Oct 19 00:44:22.535042 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-afea1fe3-3230-446f-9abe-e02cdc8f06fc tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:44:22.536349 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-afea1fe3-3230-446f-9abe-e02cdc8f06fc tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] [instance: 5bdc36c3-1db5-4ff2-808d-1295f314d522] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:44:22.548241 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-afea1fe3-3230-446f-9abe-e02cdc8f06fc tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:22.549249 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-afea1fe3-3230-446f-9abe-e02cdc8f06fc tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:22.549909 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-afea1fe3-3230-446f-9abe-e02cdc8f06fc tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:22.550998 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-afea1fe3-3230-446f-9abe-e02cdc8f06fc tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:22.551739 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-afea1fe3-3230-446f-9abe-e02cdc8f06fc tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:22.552782 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-afea1fe3-3230-446f-9abe-e02cdc8f06fc tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:22.687470 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.objects.instance [None req-afea1fe3-3230-446f-9abe-e02cdc8f06fc tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lazy-loading 'services' on Instance uuid 5bdc36c3-1db5-4ff2-808d-1295f314d522 {{(pid=125641) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 19 00:44:22.799888 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-afea1fe3-3230-446f-9abe-e02cdc8f06fc tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] 10.222.0.26 "DELETE /compute/v2.1/servers/5bdc36c3-1db5-4ff2-808d-1295f314d522/os-volume_attachments/75266677-7423-452c-84c0-e03427505bf7" status: 202 len: 0 microversion: 2.1 time: 0.277503 Oct 19 00:44:22.801142 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 330/445] 10.222.0.26 () {58 vars in 1416 bytes} [Wed Oct 19 00:44:22 2022] DELETE /compute/v2.1/servers/5bdc36c3-1db5-4ff2-808d-1295f314d522/os-volume_attachments/75266677-7423-452c-84c0-e03427505bf7 => generated 0 bytes in 280 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 19 00:44:22.812902 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-9e9f27f0-00d7-4b39-9156-9967eb286038 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:44:22.813730 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-9e9f27f0-00d7-4b39-9156-9967eb286038 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] [instance: dc8b8a6e-eab7-4f45-8f5d-916c52eec398] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:44:22.825837 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-9e9f27f0-00d7-4b39-9156-9967eb286038 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:22.826689 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-9e9f27f0-00d7-4b39-9156-9967eb286038 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:22.827677 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-9e9f27f0-00d7-4b39-9156-9967eb286038 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:22.828973 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-9e9f27f0-00d7-4b39-9156-9967eb286038 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:22.829755 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-9e9f27f0-00d7-4b39-9156-9967eb286038 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:22.831096 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-9e9f27f0-00d7-4b39-9156-9967eb286038 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:22.884721 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-9e9f27f0-00d7-4b39-9156-9967eb286038 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] [instance: dc8b8a6e-eab7-4f45-8f5d-916c52eec398] Going to try to terminate instance {{(pid=125641) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 19 00:44:23.007627 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-9e9f27f0-00d7-4b39-9156-9967eb286038 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] 10.222.0.26 "DELETE /compute/v2.1/servers/dc8b8a6e-eab7-4f45-8f5d-916c52eec398" status: 204 len: 0 microversion: 2.1 time: 0.204190 Oct 19 00:44:23.008822 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 331/446] 10.222.0.26 () {58 vars in 1239 bytes} [Wed Oct 19 00:44:22 2022] DELETE /compute/v2.1/servers/dc8b8a6e-eab7-4f45-8f5d-916c52eec398 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 19 00:44:23.022114 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-ea7dc420-dfbc-4c24-a3fe-f64e4767b0a8 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:44:23.023472 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-ea7dc420-dfbc-4c24-a3fe-f64e4767b0a8 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] [instance: dc8b8a6e-eab7-4f45-8f5d-916c52eec398] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:44:23.035185 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ea7dc420-dfbc-4c24-a3fe-f64e4767b0a8 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:23.036012 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ea7dc420-dfbc-4c24-a3fe-f64e4767b0a8 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:23.036824 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ea7dc420-dfbc-4c24-a3fe-f64e4767b0a8 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:23.038069 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ea7dc420-dfbc-4c24-a3fe-f64e4767b0a8 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:23.038925 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ea7dc420-dfbc-4c24-a3fe-f64e4767b0a8 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:23.039657 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ea7dc420-dfbc-4c24-a3fe-f64e4767b0a8 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:23.208156 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-ea7dc420-dfbc-4c24-a3fe-f64e4767b0a8 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f8725c4d05b4dd79b05b43fba15980b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0b35c2038f6b4d0a9012b004f1416c00', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:44:23.222097 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-ea7dc420-dfbc-4c24-a3fe-f64e4767b0a8 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] 10.222.0.26 "GET /compute/v2.1/servers/dc8b8a6e-eab7-4f45-8f5d-916c52eec398" status: 200 len: 1677 microversion: 2.1 time: 0.206330 Oct 19 00:44:23.223136 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 332/447] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:44:23 2022] GET /compute/v2.1/servers/dc8b8a6e-eab7-4f45-8f5d-916c52eec398 => generated 1677 bytes in 209 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 19 00:44:24.262634 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-481344ea-b1c0-43c0-ac03-f5793a833582 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:44:24.263670 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-481344ea-b1c0-43c0-ac03-f5793a833582 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] [instance: dc8b8a6e-eab7-4f45-8f5d-916c52eec398] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:44:24.271053 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-481344ea-b1c0-43c0-ac03-f5793a833582 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:24.271551 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-481344ea-b1c0-43c0-ac03-f5793a833582 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:24.271989 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-481344ea-b1c0-43c0-ac03-f5793a833582 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:24.272712 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-481344ea-b1c0-43c0-ac03-f5793a833582 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:24.273169 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-481344ea-b1c0-43c0-ac03-f5793a833582 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:24.273603 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-481344ea-b1c0-43c0-ac03-f5793a833582 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:24.428137 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-481344ea-b1c0-43c0-ac03-f5793a833582 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f8725c4d05b4dd79b05b43fba15980b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0b35c2038f6b4d0a9012b004f1416c00', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:44:24.436783 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-481344ea-b1c0-43c0-ac03-f5793a833582 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] 10.222.0.26 "GET /compute/v2.1/servers/dc8b8a6e-eab7-4f45-8f5d-916c52eec398" status: 200 len: 1677 microversion: 2.1 time: 0.178884 Oct 19 00:44:24.437400 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 333/448] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:44:24 2022] GET /compute/v2.1/servers/dc8b8a6e-eab7-4f45-8f5d-916c52eec398 => generated 1677 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 19 00:44:25.464308 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-dd66cb4b-ee0e-4992-8379-bde1d3e628e9 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:44:25.465573 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-dd66cb4b-ee0e-4992-8379-bde1d3e628e9 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] [instance: dc8b8a6e-eab7-4f45-8f5d-916c52eec398] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:44:25.471980 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-dd66cb4b-ee0e-4992-8379-bde1d3e628e9 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:25.472480 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-dd66cb4b-ee0e-4992-8379-bde1d3e628e9 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:25.472916 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-dd66cb4b-ee0e-4992-8379-bde1d3e628e9 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:25.473634 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-dd66cb4b-ee0e-4992-8379-bde1d3e628e9 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:25.474098 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-dd66cb4b-ee0e-4992-8379-bde1d3e628e9 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:25.474548 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-dd66cb4b-ee0e-4992-8379-bde1d3e628e9 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:25.716103 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-dd66cb4b-ee0e-4992-8379-bde1d3e628e9 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f8725c4d05b4dd79b05b43fba15980b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0b35c2038f6b4d0a9012b004f1416c00', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:44:25.730929 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-dd66cb4b-ee0e-4992-8379-bde1d3e628e9 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] 10.222.0.26 "GET /compute/v2.1/servers/dc8b8a6e-eab7-4f45-8f5d-916c52eec398" status: 200 len: 1677 microversion: 2.1 time: 0.270943 Oct 19 00:44:25.731985 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 334/449] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:44:25 2022] GET /compute/v2.1/servers/dc8b8a6e-eab7-4f45-8f5d-916c52eec398 => generated 1677 bytes in 273 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 19 00:44:26.431808 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [req-0c1bf682-ba88-45a8-95ce-eb2eddc09138 req-c84d0332-3683-4481-8d3d-d368601b27d8 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "dc8b8a6e-eab7-4f45-8f5d-916c52eec398", "name": "network-vif-deleted", "tag": "e00fb603-2d03-421e-b1f4-e3767c67aadc"}]} {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 19 00:44:26.459195 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-0c1bf682-ba88-45a8-95ce-eb2eddc09138 req-c84d0332-3683-4481-8d3d-d368601b27d8 service nova] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:26.460019 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-0c1bf682-ba88-45a8-95ce-eb2eddc09138 req-c84d0332-3683-4481-8d3d-d368601b27d8 service nova] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:26.460873 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-0c1bf682-ba88-45a8-95ce-eb2eddc09138 req-c84d0332-3683-4481-8d3d-d368601b27d8 service nova] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:26.488037 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.compute.server_external_events [req-0c1bf682-ba88-45a8-95ce-eb2eddc09138 req-c84d0332-3683-4481-8d3d-d368601b27d8 service nova] Creating event network-vif-deleted:e00fb603-2d03-421e-b1f4-e3767c67aadc for instance dc8b8a6e-eab7-4f45-8f5d-916c52eec398 on hv-cinder-maste Oct 19 00:44:26.502110 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [req-0c1bf682-ba88-45a8-95ce-eb2eddc09138 req-c84d0332-3683-4481-8d3d-d368601b27d8 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.081182 Oct 19 00:44:26.503232 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 335/450] 10.222.0.26 () {64 vars in 1317 bytes} [Wed Oct 19 00:44:26 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 19 00:44:26.762605 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-0a86dcb0-60fc-4712-b4b8-b74e4efeba0f tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:44:26.763344 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-0a86dcb0-60fc-4712-b4b8-b74e4efeba0f tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] [instance: dc8b8a6e-eab7-4f45-8f5d-916c52eec398] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:44:26.771180 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-0a86dcb0-60fc-4712-b4b8-b74e4efeba0f tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:26.771675 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-0a86dcb0-60fc-4712-b4b8-b74e4efeba0f tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:26.772115 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-0a86dcb0-60fc-4712-b4b8-b74e4efeba0f tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:26.773032 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-0a86dcb0-60fc-4712-b4b8-b74e4efeba0f tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:26.773490 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-0a86dcb0-60fc-4712-b4b8-b74e4efeba0f tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:26.773917 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-0a86dcb0-60fc-4712-b4b8-b74e4efeba0f tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:26.815672 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-0a86dcb0-60fc-4712-b4b8-b74e4efeba0f tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:26.816166 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-0a86dcb0-60fc-4712-b4b8-b74e4efeba0f tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:26.816593 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-0a86dcb0-60fc-4712-b4b8-b74e4efeba0f tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:26.817042 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.objects.instance [None req-0a86dcb0-60fc-4712-b4b8-b74e4efeba0f tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lazy-loading 'fault' on Instance uuid dc8b8a6e-eab7-4f45-8f5d-916c52eec398 {{(pid=125641) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 19 00:44:26.862759 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-0a86dcb0-60fc-4712-b4b8-b74e4efeba0f tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f8725c4d05b4dd79b05b43fba15980b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0b35c2038f6b4d0a9012b004f1416c00', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:44:26.872470 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-0a86dcb0-60fc-4712-b4b8-b74e4efeba0f tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] 10.222.0.26 "GET /compute/v2.1/servers/dc8b8a6e-eab7-4f45-8f5d-916c52eec398" status: 200 len: 1330 microversion: 2.1 time: 0.115672 Oct 19 00:44:26.873047 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 336/451] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:44:26 2022] GET /compute/v2.1/servers/dc8b8a6e-eab7-4f45-8f5d-916c52eec398 => generated 1330 bytes in 117 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 19 00:44:27.903202 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-c1fa877b-9f2c-4eb9-9de2-976069572178 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:44:27.904288 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-c1fa877b-9f2c-4eb9-9de2-976069572178 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] [instance: dc8b8a6e-eab7-4f45-8f5d-916c52eec398] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:44:27.914753 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-c1fa877b-9f2c-4eb9-9de2-976069572178 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:27.915740 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-c1fa877b-9f2c-4eb9-9de2-976069572178 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:27.917060 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-c1fa877b-9f2c-4eb9-9de2-976069572178 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:27.918657 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-c1fa877b-9f2c-4eb9-9de2-976069572178 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:27.920552 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-c1fa877b-9f2c-4eb9-9de2-976069572178 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:27.921669 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-c1fa877b-9f2c-4eb9-9de2-976069572178 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:27.988807 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-c1fa877b-9f2c-4eb9-9de2-976069572178 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:27.989661 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-c1fa877b-9f2c-4eb9-9de2-976069572178 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:27.990430 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-c1fa877b-9f2c-4eb9-9de2-976069572178 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:27.991259 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.objects.instance [None req-c1fa877b-9f2c-4eb9-9de2-976069572178 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lazy-loading 'fault' on Instance uuid dc8b8a6e-eab7-4f45-8f5d-916c52eec398 {{(pid=125641) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 19 00:44:28.048338 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-c1fa877b-9f2c-4eb9-9de2-976069572178 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f8725c4d05b4dd79b05b43fba15980b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0b35c2038f6b4d0a9012b004f1416c00', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:44:28.056473 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-c1fa877b-9f2c-4eb9-9de2-976069572178 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] 10.222.0.26 "GET /compute/v2.1/servers/dc8b8a6e-eab7-4f45-8f5d-916c52eec398" status: 200 len: 1330 microversion: 2.1 time: 0.162460 Oct 19 00:44:28.057074 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 337/452] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:44:27 2022] GET /compute/v2.1/servers/dc8b8a6e-eab7-4f45-8f5d-916c52eec398 => generated 1330 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 19 00:44:28.937253 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-db3e0b77-5fa7-4883-8031-ea2477c4508b tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Action: 'action', calling method: >, body: {"rescue": {"adminPass": "***"}} {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 19 00:44:28.939691 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-db3e0b77-5fa7-4883-8031-ea2477c4508b tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] [instance: f6051269-1d16-42db-8323-67a532dbcf40] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:44:28.946774 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-db3e0b77-5fa7-4883-8031-ea2477c4508b tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:28.947316 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-db3e0b77-5fa7-4883-8031-ea2477c4508b tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:28.948653 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-db3e0b77-5fa7-4883-8031-ea2477c4508b tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:28.949725 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-db3e0b77-5fa7-4883-8031-ea2477c4508b tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:28.950459 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-db3e0b77-5fa7-4883-8031-ea2477c4508b tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:28.951054 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-db3e0b77-5fa7-4883-8031-ea2477c4508b tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:28.977938 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.objects.instance [None req-db3e0b77-5fa7-4883-8031-ea2477c4508b tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lazy-loading 'flavor' on Instance uuid f6051269-1d16-42db-8323-67a532dbcf40 {{(pid=125641) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 19 00:44:29.171274 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-db3e0b77-5fa7-4883-8031-ea2477c4508b tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] 10.222.0.26 "POST /compute/v2.1/servers/f6051269-1d16-42db-8323-67a532dbcf40/action" status: 200 len: 32 microversion: 2.1 time: 0.237561 Oct 19 00:44:29.172389 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 338/453] 10.222.0.26 () {60 vars in 1278 bytes} [Wed Oct 19 00:44:28 2022] POST /compute/v2.1/servers/f6051269-1d16-42db-8323-67a532dbcf40/action => generated 32 bytes in 240 msecs (HTTP/1.1 200) 9 headers in 356 bytes (2 switches on core 0) Oct 19 00:44:29.180810 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-987ef77b-413d-4a1f-9c05-8a50f3a9449b tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:44:29.181541 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-987ef77b-413d-4a1f-9c05-8a50f3a9449b tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] [instance: dc8b8a6e-eab7-4f45-8f5d-916c52eec398] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:44:29.193268 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-987ef77b-413d-4a1f-9c05-8a50f3a9449b tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:29.194132 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-987ef77b-413d-4a1f-9c05-8a50f3a9449b tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:29.194951 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-987ef77b-413d-4a1f-9c05-8a50f3a9449b tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:29.196306 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-987ef77b-413d-4a1f-9c05-8a50f3a9449b tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:29.197310 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-987ef77b-413d-4a1f-9c05-8a50f3a9449b tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:29.198615 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-987ef77b-413d-4a1f-9c05-8a50f3a9449b tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:29.206989 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG dbcounter [-] [125641] Writing DB stats nova_cell1:SELECT=92,nova_cell1:UPDATE=4,nova_cell1:INSERT=4 {{(pid=125641) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 00:44:29.221474 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.wsgi [None req-987ef77b-413d-4a1f-9c05-8a50f3a9449b tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] HTTP exception thrown: Instance dc8b8a6e-eab7-4f45-8f5d-916c52eec398 could not be found. Oct 19 00:44:29.223176 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-987ef77b-413d-4a1f-9c05-8a50f3a9449b tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Request method failure captured: Oct 19 00:44:29.223176 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: request: GET /compute/v2.1/servers/dc8b8a6e-eab7-4f45-8f5d-916c52eec398 HTTP/1.1 Oct 19 00:44:29.223176 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Accept: application/json Oct 19 00:44:29.223176 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Accept-Encoding: identity Oct 19 00:44:29.223176 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Connection: close Oct 19 00:44:29.223176 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Content-Length: 0 Oct 19 00:44:29.223176 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Content-Type: application/json Oct 19 00:44:29.223176 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Host: 10.222.0.26 Oct 19 00:44:29.223176 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Openstack-System-Scope: None Oct 19 00:44:29.223176 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: User-Agent: python-urllib3/1.26.12 Oct 19 00:44:29.223176 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Auth-Token: gAAAAABjT0eJXd9ar1txWHw929SSSa4BnzN-f-jxcCDsiqb5Qu6eweguqGUJXghFM8xaTx1ciC3SJFoac87bZHlU5VANYjjzV6ICQChedig2cB1RmNul9h3AYlunCOiAit1Pl73UlpO5HCAxVXGmz0N3G59VsZacZSMYF6RytJzlde2rg7lCpks Oct 19 00:44:29.223176 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Domain-Id: None Oct 19 00:44:29.223176 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Domain-Name: None Oct 19 00:44:29.223176 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Identity-Status: Confirmed Oct 19 00:44:29.223176 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Is-Admin-Project: True Oct 19 00:44:29.223176 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Project-Domain-Id: default Oct 19 00:44:29.223176 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Project-Domain-Name: Default Oct 19 00:44:29.223176 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Project-Id: 0b35c2038f6b4d0a9012b004f1416c00 Oct 19 00:44:29.223176 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Project-Name: tempest-AttachVolumeNegativeTest-1282551484 Oct 19 00:44:29.223176 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Role: member,reader Oct 19 00:44:29.223176 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Roles: member,reader Oct 19 00:44:29.231909 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/0b35c2038f6b4d0a9012b004f1416c00"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/0b35c2038f6b4d0a9012b004f1416c00"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/0b35c2038f6b4d0a9012b004f1416c00"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_0b35c2038f6b4d0a9012b004f1416c00"}]}] Oct 19 00:44:29.231909 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Tenant: tempest-AttachVolumeNegativeTest-1282551484 Oct 19 00:44:29.231909 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Tenant-Id: 0b35c2038f6b4d0a9012b004f1416c00 Oct 19 00:44:29.231909 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1282551484 Oct 19 00:44:29.231909 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User: tempest-AttachVolumeNegativeTest-1282551484-project Oct 19 00:44:29.231909 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User-Domain-Id: default Oct 19 00:44:29.231909 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User-Domain-Name: Default Oct 19 00:44:29.231909 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User-Id: 6f8725c4d05b4dd79b05b43fba15980b Oct 19 00:44:29.231909 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User-Name: tempest-AttachVolumeNegativeTest-1282551484-project Oct 19 00:44:29.231909 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: method: > Oct 19 00:44:29.231909 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: exception: Instance dc8b8a6e-eab7-4f45-8f5d-916c52eec398 could not be found. Oct 19 00:44:29.231909 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: action_args: {'id': 'dc8b8a6e-eab7-4f45-8f5d-916c52eec398'} Oct 19 00:44:29.233668 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 19 00:44:29.233668 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 19 00:44:29.233668 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 19 00:44:29.233668 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 19 00:44:29.233668 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 19 00:44:29.233668 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 19 00:44:29.233668 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 19 00:44:29.233668 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 19 00:44:29.233668 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 19 00:44:29.233668 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 19 00:44:29.233668 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance dc8b8a6e-eab7-4f45-8f5d-916c52eec398 could not be found. Oct 19 00:44:29.233668 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:44:29.233668 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 19 00:44:29.233668 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:44:29.233668 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 19 00:44:29.233668 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 19 00:44:29.233668 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 19 00:44:29.233668 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 19 00:44:29.233668 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 19 00:44:29.233668 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 19 00:44:29.236125 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 19 00:44:29.236125 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 19 00:44:29.236125 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 19 00:44:29.236125 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 19 00:44:29.236125 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 19 00:44:29.236125 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 19 00:44:29.236125 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 19 00:44:29.236125 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance dc8b8a6e-eab7-4f45-8f5d-916c52eec398 could not be found. Oct 19 00:44:29.236125 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:44:29.236125 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 19 00:44:29.236125 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:44:29.236125 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 19 00:44:29.236125 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 19 00:44:29.236125 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 19 00:44:29.236125 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 19 00:44:29.236125 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 19 00:44:29.236125 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance dc8b8a6e-eab7-4f45-8f5d-916c52eec398 could not be found. Oct 19 00:44:29.236125 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:44:29.238533 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-987ef77b-413d-4a1f-9c05-8a50f3a9449b tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Returning 404 to user: Instance dc8b8a6e-eab7-4f45-8f5d-916c52eec398 could not be found. {{(pid=125641) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 19 00:44:29.238533 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-987ef77b-413d-4a1f-9c05-8a50f3a9449b tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] 10.222.0.26 "GET /compute/v2.1/servers/dc8b8a6e-eab7-4f45-8f5d-916c52eec398" status: 404 len: 111 microversion: 2.1 time: 0.050546 Oct 19 00:44:29.238533 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 339/454] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:44:29 2022] GET /compute/v2.1/servers/dc8b8a6e-eab7-4f45-8f5d-916c52eec398 => generated 111 bytes in 52 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 19 00:44:29.238533 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-ffc32dbc-6ff3-441d-9d44-03529a72ee90 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:44:29.238533 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-ffc32dbc-6ff3-441d-9d44-03529a72ee90 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] [instance: f6051269-1d16-42db-8323-67a532dbcf40] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:44:29.239637 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ffc32dbc-6ff3-441d-9d44-03529a72ee90 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:29.239637 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ffc32dbc-6ff3-441d-9d44-03529a72ee90 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:29.239637 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ffc32dbc-6ff3-441d-9d44-03529a72ee90 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:29.240387 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ffc32dbc-6ff3-441d-9d44-03529a72ee90 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:29.240589 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ffc32dbc-6ff3-441d-9d44-03529a72ee90 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:29.241017 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ffc32dbc-6ff3-441d-9d44-03529a72ee90 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:29.475797 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-ffc32dbc-6ff3-441d-9d44-03529a72ee90 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd3f0d615187d4779bdd7da8c9f24c796', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0151d58abd047a692f5cc4e7d05bea5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:44:29.505625 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-ffc32dbc-6ff3-441d-9d44-03529a72ee90 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] 10.222.0.26 "GET /compute/v2.1/servers/f6051269-1d16-42db-8323-67a532dbcf40" status: 200 len: 1737 microversion: 2.1 time: 0.278572 Oct 19 00:44:29.506940 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 340/455] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:44:29 2022] GET /compute/v2.1/servers/f6051269-1d16-42db-8323-67a532dbcf40 => generated 1737 bytes in 281 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:44:29.516860 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-b9de13cd-cb5f-4ee1-9495-2ff4bb738445 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Calling method '.version_select at 0x7f71c3ff11f0>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:44:29.553601 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-b9de13cd-cb5f-4ee1-9495-2ff4bb738445 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] 10.222.0.26 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-80676479" status: 202 len: 0 microversion: 2.1 time: 0.044677 Oct 19 00:44:29.559613 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 341/456] 10.222.0.26 () {58 vars in 1215 bytes} [Wed Oct 19 00:44:29 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-80676479 => generated 0 bytes in 48 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 19 00:44:30.535480 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-ba596d4c-4ad9-4836-9619-719cd53c942d tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:44:30.536536 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-ba596d4c-4ad9-4836-9619-719cd53c942d tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] [instance: f6051269-1d16-42db-8323-67a532dbcf40] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:44:30.546201 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ba596d4c-4ad9-4836-9619-719cd53c942d tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:30.547017 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ba596d4c-4ad9-4836-9619-719cd53c942d tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:30.547774 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ba596d4c-4ad9-4836-9619-719cd53c942d tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:30.549765 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ba596d4c-4ad9-4836-9619-719cd53c942d tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:30.551955 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ba596d4c-4ad9-4836-9619-719cd53c942d tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:30.552860 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ba596d4c-4ad9-4836-9619-719cd53c942d tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:30.742218 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-ba596d4c-4ad9-4836-9619-719cd53c942d tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd3f0d615187d4779bdd7da8c9f24c796', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0151d58abd047a692f5cc4e7d05bea5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:44:30.752089 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-ba596d4c-4ad9-4836-9619-719cd53c942d tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] 10.222.0.26 "GET /compute/v2.1/servers/f6051269-1d16-42db-8323-67a532dbcf40" status: 200 len: 1737 microversion: 2.1 time: 0.223809 Oct 19 00:44:30.752684 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 342/457] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:44:30 2022] GET /compute/v2.1/servers/f6051269-1d16-42db-8323-67a532dbcf40 => generated 1737 bytes in 225 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:44:30.758007 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-5976dcf8-cd49-44e1-b2b7-099fe4e8f95c tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:44:30.758577 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-5976dcf8-cd49-44e1-b2b7-099fe4e8f95c tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] [instance: 5bdc36c3-1db5-4ff2-808d-1295f314d522] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:44:30.766232 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-5976dcf8-cd49-44e1-b2b7-099fe4e8f95c tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:30.766799 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-5976dcf8-cd49-44e1-b2b7-099fe4e8f95c tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:30.767317 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-5976dcf8-cd49-44e1-b2b7-099fe4e8f95c tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:30.768089 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-5976dcf8-cd49-44e1-b2b7-099fe4e8f95c tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:30.768580 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-5976dcf8-cd49-44e1-b2b7-099fe4e8f95c tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:30.769035 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-5976dcf8-cd49-44e1-b2b7-099fe4e8f95c tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:30.815901 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-5976dcf8-cd49-44e1-b2b7-099fe4e8f95c tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] [instance: 5bdc36c3-1db5-4ff2-808d-1295f314d522] Going to try to terminate instance {{(pid=125641) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 19 00:44:30.918417 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-5976dcf8-cd49-44e1-b2b7-099fe4e8f95c tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] 10.222.0.26 "DELETE /compute/v2.1/servers/5bdc36c3-1db5-4ff2-808d-1295f314d522" status: 204 len: 0 microversion: 2.1 time: 0.164317 Oct 19 00:44:30.919468 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 343/458] 10.222.0.26 () {58 vars in 1239 bytes} [Wed Oct 19 00:44:30 2022] DELETE /compute/v2.1/servers/5bdc36c3-1db5-4ff2-808d-1295f314d522 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 19 00:44:30.926143 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-3cc3d09a-4c7c-4e0a-a522-c4f8fb34f071 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Action: 'create', calling method: .version_select at 0x7f71c3f25f70>, body: {"keypair": {"name": "tempest-keypair-1805314021", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBMcBLKwhB01pmue6ZvHN5pRjLPiP56l+fuBRzOkjHgnYpTypfIrkVMMGftkUm+6Rjov4ZHszAr3kdyck8AGKdEFwk1roA2OiLHEG7e4ffHtTzvrs66hxAk30XtXsLBOgUA=="}} {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 19 00:44:30.948316 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.quota [None req-3cc3d09a-4c7c-4e0a-a522-c4f8fb34f071 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Getting quotas for project 0b35c2038f6b4d0a9012b004f1416c00. Resources: {'key_pairs'} {{(pid=125641) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 19 00:44:30.954672 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.quota [None req-3cc3d09a-4c7c-4e0a-a522-c4f8fb34f071 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Getting quotas for user 6f8725c4d05b4dd79b05b43fba15980b and project 0b35c2038f6b4d0a9012b004f1416c00. Resources: {'key_pairs'} {{(pid=125641) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 19 00:44:30.997860 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-3cc3d09a-4c7c-4e0a-a522-c4f8fb34f071 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] 10.222.0.26 "POST /compute/v2.1/os-keypairs" status: 200 len: 386 microversion: 2.1 time: 0.076384 Oct 19 00:44:30.999005 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 344/459] 10.222.0.26 () {60 vars in 1159 bytes} [Wed Oct 19 00:44:30 2022] POST /compute/v2.1/os-keypairs => generated 386 bytes in 79 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 19 00:44:31.007873 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-9c5251ea-2369-490c-bcb4-668d16036088 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:44:31.009230 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-9c5251ea-2369-490c-bcb4-668d16036088 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] [instance: 5bdc36c3-1db5-4ff2-808d-1295f314d522] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:44:31.022370 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-9c5251ea-2369-490c-bcb4-668d16036088 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:31.023244 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-9c5251ea-2369-490c-bcb4-668d16036088 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:31.023998 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-9c5251ea-2369-490c-bcb4-668d16036088 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:31.025270 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-9c5251ea-2369-490c-bcb4-668d16036088 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:31.027058 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-9c5251ea-2369-490c-bcb4-668d16036088 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:31.027944 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-9c5251ea-2369-490c-bcb4-668d16036088 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:31.259032 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-9c5251ea-2369-490c-bcb4-668d16036088 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '39de6a6fff7c4cc08cc62aa86182d71d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1235d8b054b3405595fca761f08b73ae', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:44:31.267838 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-9c5251ea-2369-490c-bcb4-668d16036088 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] 10.222.0.26 "GET /compute/v2.1/servers/5bdc36c3-1db5-4ff2-808d-1295f314d522" status: 200 len: 1514 microversion: 2.1 time: 0.267305 Oct 19 00:44:31.270523 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 345/460] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:44:30 2022] GET /compute/v2.1/servers/5bdc36c3-1db5-4ff2-808d-1295f314d522 => generated 1514 bytes in 271 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:44:31.705959 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-94f73b78-0d60-4c41-a91b-b1c01f44ef48 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestVolumeBootPattern-server-782545817", "imageRef": "", "flavorRef": "452", "key_name": "tempest-TestVolumeBootPattern-1204843348", "security_groups": [{"name": "tempest-secgroup-smoke-1056832705"}], "block_device_mapping_v2": [{"uuid": "c5018c37-69d3-44f9-ac3c-7a8377379ade", "source_type": "volume", "destination_type": "volume", "boot_index": 0, "delete_on_termination": false}], "networks": [{"uuid": "cfe47441-dd27-4b7c-958a-cc2e5c8243ac"}]}} {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 19 00:44:31.857391 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.network.neutron [None req-94f73b78-0d60-4c41-a91b-b1c01f44ef48 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] validate_networks() for [('cfe47441-dd27-4b7c-958a-cc2e5c8243ac', None, None, None, None, None)] {{(pid=125641) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 19 00:44:32.397844 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.virt.hardware [None req-94f73b78-0d60-4c41-a91b-b1c01f44ef48 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Flavor limits 0:0:0 {{(pid=125641) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 19 00:44:32.398665 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.virt.hardware [None req-94f73b78-0d60-4c41-a91b-b1c01f44ef48 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Image limits 0:0:0 {{(pid=125641) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 19 00:44:32.399603 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.virt.hardware [None req-94f73b78-0d60-4c41-a91b-b1c01f44ef48 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Flavor pref 0:0:0 {{(pid=125641) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 19 00:44:32.400443 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.virt.hardware [None req-94f73b78-0d60-4c41-a91b-b1c01f44ef48 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Image pref 0:0:0 {{(pid=125641) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 19 00:44:32.402431 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.virt.hardware [None req-94f73b78-0d60-4c41-a91b-b1c01f44ef48 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=125641) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 19 00:44:32.419301 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-94f73b78-0d60-4c41-a91b-b1c01f44ef48 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:32.420157 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-94f73b78-0d60-4c41-a91b-b1c01f44ef48 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:32.421118 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-94f73b78-0d60-4c41-a91b-b1c01f44ef48 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:32.422579 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-94f73b78-0d60-4c41-a91b-b1c01f44ef48 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:32.423131 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-94f73b78-0d60-4c41-a91b-b1c01f44ef48 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:32.423608 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-94f73b78-0d60-4c41-a91b-b1c01f44ef48 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:32.452637 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.quota [None req-94f73b78-0d60-4c41-a91b-b1c01f44ef48 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Getting quotas for project 8684504b0eff415b9063cebae21ca667. Resources: {'cores', 'ram', 'instances'} {{(pid=125641) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 19 00:44:32.459893 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.quota [None req-94f73b78-0d60-4c41-a91b-b1c01f44ef48 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Getting quotas for user f738e67d8c2340028da1254b60d7810e and project 8684504b0eff415b9063cebae21ca667. Resources: {'cores', 'ram', 'instances'} {{(pid=125641) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 19 00:44:32.532978 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-94f73b78-0d60-4c41-a91b-b1c01f44ef48 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Going to run 1 instances... {{(pid=125641) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 19 00:44:32.535433 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-94f73b78-0d60-4c41-a91b-b1c01f44ef48 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] [instance: 0b7c19ee-6d89-4855-bf4b-d35b65bb170b] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=False,deleted=,deleted_at=,destination_type='volume',device_name=None,device_type=None,disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id=None,instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='volume',tag=None,updated_at=,uuid=,volume_id='c5018c37-69d3-44f9-ac3c-7a8377379ade',volume_size=None,volume_type=None)] {{(pid=125641) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 19 00:44:32.708143 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-94f73b78-0d60-4c41-a91b-b1c01f44ef48 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 202 len: 406 microversion: 2.1 time: 1.007294 Oct 19 00:44:32.710422 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 346/461] 10.222.0.26 () {60 vars in 1147 bytes} [Wed Oct 19 00:44:31 2022] POST /compute/v2.1/servers => generated 406 bytes in 1011 msecs (HTTP/1.1 202) 10 headers in 451 bytes (2 switches on core 0) Oct 19 00:44:32.720137 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-dc043169-ab9a-4a1a-ac7b-727da8edaa8e tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:44:32.721668 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-dc043169-ab9a-4a1a-ac7b-727da8edaa8e tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] [instance: f6051269-1d16-42db-8323-67a532dbcf40] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:44:32.743615 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-dc043169-ab9a-4a1a-ac7b-727da8edaa8e tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:32.745123 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-dc043169-ab9a-4a1a-ac7b-727da8edaa8e tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:32.746289 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-dc043169-ab9a-4a1a-ac7b-727da8edaa8e tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:32.747701 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-dc043169-ab9a-4a1a-ac7b-727da8edaa8e tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:32.749228 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-dc043169-ab9a-4a1a-ac7b-727da8edaa8e tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:32.751667 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-dc043169-ab9a-4a1a-ac7b-727da8edaa8e tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:33.037659 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-dc043169-ab9a-4a1a-ac7b-727da8edaa8e tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd3f0d615187d4779bdd7da8c9f24c796', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0151d58abd047a692f5cc4e7d05bea5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:44:33.058185 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-dc043169-ab9a-4a1a-ac7b-727da8edaa8e tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] 10.222.0.26 "GET /compute/v2.1/servers/f6051269-1d16-42db-8323-67a532dbcf40" status: 200 len: 1737 microversion: 2.1 time: 0.344631 Oct 19 00:44:33.059236 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 347/462] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:44:32 2022] GET /compute/v2.1/servers/f6051269-1d16-42db-8323-67a532dbcf40 => generated 1737 bytes in 348 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:44:33.068672 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-357c9197-182c-4fb3-8370-c039ab0865bf tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:44:33.070179 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-357c9197-182c-4fb3-8370-c039ab0865bf tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] [instance: 5bdc36c3-1db5-4ff2-808d-1295f314d522] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:44:33.081983 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-357c9197-182c-4fb3-8370-c039ab0865bf tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:33.083515 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-357c9197-182c-4fb3-8370-c039ab0865bf tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:33.084333 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-357c9197-182c-4fb3-8370-c039ab0865bf tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:33.085629 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-357c9197-182c-4fb3-8370-c039ab0865bf tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:33.086544 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-357c9197-182c-4fb3-8370-c039ab0865bf tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:33.090018 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-357c9197-182c-4fb3-8370-c039ab0865bf tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:33.328938 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-357c9197-182c-4fb3-8370-c039ab0865bf tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '39de6a6fff7c4cc08cc62aa86182d71d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1235d8b054b3405595fca761f08b73ae', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:44:33.351595 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-357c9197-182c-4fb3-8370-c039ab0865bf tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] 10.222.0.26 "GET /compute/v2.1/servers/5bdc36c3-1db5-4ff2-808d-1295f314d522" status: 200 len: 1514 microversion: 2.1 time: 0.290212 Oct 19 00:44:33.353208 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 348/463] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:44:33 2022] GET /compute/v2.1/servers/5bdc36c3-1db5-4ff2-808d-1295f314d522 => generated 1514 bytes in 293 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:44:33.365117 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-5f203c06-7c96-4a38-93f5-279620fc72bf tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:44:33.366877 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-5f203c06-7c96-4a38-93f5-279620fc72bf tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] [instance: 0b7c19ee-6d89-4855-bf4b-d35b65bb170b] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:44:33.384641 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-5f203c06-7c96-4a38-93f5-279620fc72bf tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:33.386048 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-5f203c06-7c96-4a38-93f5-279620fc72bf tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:33.387056 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-5f203c06-7c96-4a38-93f5-279620fc72bf tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:33.388358 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-5f203c06-7c96-4a38-93f5-279620fc72bf tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:33.389438 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-5f203c06-7c96-4a38-93f5-279620fc72bf tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:33.390574 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-5f203c06-7c96-4a38-93f5-279620fc72bf tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:33.506973 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-5f203c06-7c96-4a38-93f5-279620fc72bf tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f738e67d8c2340028da1254b60d7810e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8684504b0eff415b9063cebae21ca667', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:44:33.524930 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-5f203c06-7c96-4a38-93f5-279620fc72bf tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] 10.222.0.26 "GET /compute/v2.1/servers/0b7c19ee-6d89-4855-bf4b-d35b65bb170b" status: 200 len: 1144 microversion: 2.1 time: 0.169249 Oct 19 00:44:33.526005 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 349/464] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:44:33 2022] GET /compute/v2.1/servers/0b7c19ee-6d89-4855-bf4b-d35b65bb170b => generated 1144 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:44:33.535173 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-56d82a1b-c6e0-4309-898e-1910e77bd13a tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeNegativeTest-server-1181346950", "imageRef": "b387b4f8-f75e-4a68-b49c-a015bf61c46b", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "635fade4-7ee9-45f4-8020-ceb7d6bf784a"}], "security_groups": [{"name": "tempest-securitygroup--176955835"}], "key_name": "tempest-keypair-1805314021", "user_data": "}} {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 19 00:44:33.741932 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.network.neutron [None req-56d82a1b-c6e0-4309-898e-1910e77bd13a tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] validate_networks() for [('635fade4-7ee9-45f4-8020-ceb7d6bf784a', None, None, None, None, None)] {{(pid=125641) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 19 00:44:34.309636 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.virt.hardware [None req-56d82a1b-c6e0-4309-898e-1910e77bd13a tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Flavor limits 0:0:0 {{(pid=125641) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 19 00:44:34.310153 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.virt.hardware [None req-56d82a1b-c6e0-4309-898e-1910e77bd13a tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Image limits 0:0:0 {{(pid=125641) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 19 00:44:34.311087 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.virt.hardware [None req-56d82a1b-c6e0-4309-898e-1910e77bd13a tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Flavor pref 0:0:0 {{(pid=125641) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 19 00:44:34.311668 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.virt.hardware [None req-56d82a1b-c6e0-4309-898e-1910e77bd13a tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Image pref 0:0:0 {{(pid=125641) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 19 00:44:34.312123 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.virt.hardware [None req-56d82a1b-c6e0-4309-898e-1910e77bd13a tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=125641) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 19 00:44:34.322797 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-56d82a1b-c6e0-4309-898e-1910e77bd13a tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:34.323666 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-56d82a1b-c6e0-4309-898e-1910e77bd13a tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:34.324357 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-56d82a1b-c6e0-4309-898e-1910e77bd13a tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:34.325424 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-56d82a1b-c6e0-4309-898e-1910e77bd13a tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:34.326039 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-56d82a1b-c6e0-4309-898e-1910e77bd13a tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:34.326460 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-56d82a1b-c6e0-4309-898e-1910e77bd13a tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:34.347420 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG dbcounter [-] [125641] Writing DB stats nova_api:SELECT=93,nova_api:UPDATE=2,nova_api:DELETE=1,nova_api:INSERT=4 {{(pid=125641) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 00:44:34.358862 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.quota [None req-56d82a1b-c6e0-4309-898e-1910e77bd13a tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Getting quotas for project 0b35c2038f6b4d0a9012b004f1416c00. Resources: {'cores', 'ram', 'instances'} {{(pid=125641) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 19 00:44:34.365382 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.quota [None req-56d82a1b-c6e0-4309-898e-1910e77bd13a tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Getting quotas for user 6f8725c4d05b4dd79b05b43fba15980b and project 0b35c2038f6b4d0a9012b004f1416c00. Resources: {'cores', 'ram', 'instances'} {{(pid=125641) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 19 00:44:34.385276 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-56d82a1b-c6e0-4309-898e-1910e77bd13a tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Going to run 1 instances... {{(pid=125641) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 19 00:44:34.388516 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-56d82a1b-c6e0-4309-898e-1910e77bd13a tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] [instance: d8ff0e88-597f-4db1-b5f4-852dd33c566d] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='b387b4f8-f75e-4a68-b49c-a015bf61c46b',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=125641) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 19 00:44:34.467050 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-56d82a1b-c6e0-4309-898e-1910e77bd13a tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 202 len: 401 microversion: 2.1 time: 0.938511 Oct 19 00:44:34.469085 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 350/465] 10.222.0.26 () {60 vars in 1147 bytes} [Wed Oct 19 00:44:33 2022] POST /compute/v2.1/servers => generated 401 bytes in 942 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 19 00:44:34.480833 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [req-16a804de-06c3-4e43-9a76-7f2274fd64c0 req-3cfbac5a-4fa9-4eea-9ac1-a8b9a972f691 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "5bdc36c3-1db5-4ff2-808d-1295f314d522", "name": "network-vif-deleted", "tag": "97ea2bef-8a45-4c17-8415-e62d303621cb"}]} {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 19 00:44:34.506267 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-16a804de-06c3-4e43-9a76-7f2274fd64c0 req-3cfbac5a-4fa9-4eea-9ac1-a8b9a972f691 service nova] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:34.507212 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-16a804de-06c3-4e43-9a76-7f2274fd64c0 req-3cfbac5a-4fa9-4eea-9ac1-a8b9a972f691 service nova] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:34.510343 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-16a804de-06c3-4e43-9a76-7f2274fd64c0 req-3cfbac5a-4fa9-4eea-9ac1-a8b9a972f691 service nova] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:34.537083 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.compute.server_external_events [req-16a804de-06c3-4e43-9a76-7f2274fd64c0 req-3cfbac5a-4fa9-4eea-9ac1-a8b9a972f691 service nova] Creating event network-vif-deleted:97ea2bef-8a45-4c17-8415-e62d303621cb for instance 5bdc36c3-1db5-4ff2-808d-1295f314d522 on hv-cinder-maste Oct 19 00:44:34.567325 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [req-16a804de-06c3-4e43-9a76-7f2274fd64c0 req-3cfbac5a-4fa9-4eea-9ac1-a8b9a972f691 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.094668 Oct 19 00:44:34.573057 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 351/466] 10.222.0.26 () {64 vars in 1317 bytes} [Wed Oct 19 00:44:34 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 19 00:44:34.589804 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-f46e6a38-c8da-4137-b824-c5319aa46254 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:44:34.591103 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-f46e6a38-c8da-4137-b824-c5319aa46254 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] [instance: f6051269-1d16-42db-8323-67a532dbcf40] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:44:34.609649 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-f46e6a38-c8da-4137-b824-c5319aa46254 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:34.621793 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-f46e6a38-c8da-4137-b824-c5319aa46254 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:34.624214 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-f46e6a38-c8da-4137-b824-c5319aa46254 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.012s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:34.627151 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-f46e6a38-c8da-4137-b824-c5319aa46254 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:34.629528 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-f46e6a38-c8da-4137-b824-c5319aa46254 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:34.630317 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-f46e6a38-c8da-4137-b824-c5319aa46254 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:34.912303 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-f46e6a38-c8da-4137-b824-c5319aa46254 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd3f0d615187d4779bdd7da8c9f24c796', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0151d58abd047a692f5cc4e7d05bea5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:44:34.940822 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-f46e6a38-c8da-4137-b824-c5319aa46254 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] 10.222.0.26 "GET /compute/v2.1/servers/f6051269-1d16-42db-8323-67a532dbcf40" status: 200 len: 1737 microversion: 2.1 time: 0.360859 Oct 19 00:44:34.955533 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 352/467] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:44:34 2022] GET /compute/v2.1/servers/f6051269-1d16-42db-8323-67a532dbcf40 => generated 1737 bytes in 380 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:44:34.979566 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-350d6fda-4a5d-4b16-b5be-dd6326ab48a9 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:44:34.980397 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-350d6fda-4a5d-4b16-b5be-dd6326ab48a9 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] [instance: 5bdc36c3-1db5-4ff2-808d-1295f314d522] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:44:35.001737 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-350d6fda-4a5d-4b16-b5be-dd6326ab48a9 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:35.002266 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-350d6fda-4a5d-4b16-b5be-dd6326ab48a9 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:35.002754 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-350d6fda-4a5d-4b16-b5be-dd6326ab48a9 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:35.003478 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-350d6fda-4a5d-4b16-b5be-dd6326ab48a9 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:35.003996 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-350d6fda-4a5d-4b16-b5be-dd6326ab48a9 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:35.004449 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-350d6fda-4a5d-4b16-b5be-dd6326ab48a9 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:35.051470 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-350d6fda-4a5d-4b16-b5be-dd6326ab48a9 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:35.052241 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-350d6fda-4a5d-4b16-b5be-dd6326ab48a9 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:35.052912 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-350d6fda-4a5d-4b16-b5be-dd6326ab48a9 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:35.053751 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.objects.instance [None req-350d6fda-4a5d-4b16-b5be-dd6326ab48a9 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lazy-loading 'fault' on Instance uuid 5bdc36c3-1db5-4ff2-808d-1295f314d522 {{(pid=125641) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 19 00:44:35.133353 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-350d6fda-4a5d-4b16-b5be-dd6326ab48a9 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '39de6a6fff7c4cc08cc62aa86182d71d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1235d8b054b3405595fca761f08b73ae', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:44:35.147267 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-350d6fda-4a5d-4b16-b5be-dd6326ab48a9 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] 10.222.0.26 "GET /compute/v2.1/servers/5bdc36c3-1db5-4ff2-808d-1295f314d522" status: 200 len: 1309 microversion: 2.1 time: 0.189987 Oct 19 00:44:35.149521 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 353/468] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:44:34 2022] GET /compute/v2.1/servers/5bdc36c3-1db5-4ff2-808d-1295f314d522 => generated 1309 bytes in 193 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:44:35.161113 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-5d302a58-0ab0-44f4-a2da-3e3ef4e6c1f8 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:44:35.162637 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-5d302a58-0ab0-44f4-a2da-3e3ef4e6c1f8 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] [instance: d8ff0e88-597f-4db1-b5f4-852dd33c566d] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:44:35.279861 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-5d302a58-0ab0-44f4-a2da-3e3ef4e6c1f8 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f8725c4d05b4dd79b05b43fba15980b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0b35c2038f6b4d0a9012b004f1416c00', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:44:35.287447 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-5d302a58-0ab0-44f4-a2da-3e3ef4e6c1f8 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] 10.222.0.26 "GET /compute/v2.1/servers/d8ff0e88-597f-4db1-b5f4-852dd33c566d" status: 200 len: 1242 microversion: 2.1 time: 0.136136 Oct 19 00:44:35.291813 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 354/469] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:44:35 2022] GET /compute/v2.1/servers/d8ff0e88-597f-4db1-b5f4-852dd33c566d => generated 1242 bytes in 142 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:44:35.307985 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-471ed87d-1d9c-4f7a-aa44-fa22b2a5f552 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:44:35.310171 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-471ed87d-1d9c-4f7a-aa44-fa22b2a5f552 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] [instance: 0b7c19ee-6d89-4855-bf4b-d35b65bb170b] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:44:35.324546 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-471ed87d-1d9c-4f7a-aa44-fa22b2a5f552 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:35.325086 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-471ed87d-1d9c-4f7a-aa44-fa22b2a5f552 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:35.325947 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-471ed87d-1d9c-4f7a-aa44-fa22b2a5f552 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:35.326891 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-471ed87d-1d9c-4f7a-aa44-fa22b2a5f552 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:35.327374 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-471ed87d-1d9c-4f7a-aa44-fa22b2a5f552 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:35.328108 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-471ed87d-1d9c-4f7a-aa44-fa22b2a5f552 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:35.406984 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-471ed87d-1d9c-4f7a-aa44-fa22b2a5f552 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f738e67d8c2340028da1254b60d7810e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8684504b0eff415b9063cebae21ca667', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:44:35.438011 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-471ed87d-1d9c-4f7a-aa44-fa22b2a5f552 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] 10.222.0.26 "GET /compute/v2.1/servers/0b7c19ee-6d89-4855-bf4b-d35b65bb170b" status: 200 len: 1210 microversion: 2.1 time: 0.140806 Oct 19 00:44:35.439110 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 355/470] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:44:35 2022] GET /compute/v2.1/servers/0b7c19ee-6d89-4855-bf4b-d35b65bb170b => generated 1210 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:44:35.988822 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-a9c29234-0b54-4d35-899f-f7cd97855ace tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:44:35.995592 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-a9c29234-0b54-4d35-899f-f7cd97855ace tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] [instance: f6051269-1d16-42db-8323-67a532dbcf40] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:44:36.010608 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-a9c29234-0b54-4d35-899f-f7cd97855ace tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:36.011692 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-a9c29234-0b54-4d35-899f-f7cd97855ace tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:36.012450 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-a9c29234-0b54-4d35-899f-f7cd97855ace tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:36.013993 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-a9c29234-0b54-4d35-899f-f7cd97855ace tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:36.014851 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-a9c29234-0b54-4d35-899f-f7cd97855ace tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:36.015831 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-a9c29234-0b54-4d35-899f-f7cd97855ace tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:36.262979 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-a9c29234-0b54-4d35-899f-f7cd97855ace tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd3f0d615187d4779bdd7da8c9f24c796', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0151d58abd047a692f5cc4e7d05bea5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:44:36.272242 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-a9c29234-0b54-4d35-899f-f7cd97855ace tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] 10.222.0.26 "GET /compute/v2.1/servers/f6051269-1d16-42db-8323-67a532dbcf40" status: 200 len: 1737 microversion: 2.1 time: 0.309956 Oct 19 00:44:36.272850 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 356/471] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:44:35 2022] GET /compute/v2.1/servers/f6051269-1d16-42db-8323-67a532dbcf40 => generated 1737 bytes in 312 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:44:36.281953 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-ce50ae4c-b4e6-4098-aa9f-3b462e3af938 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:44:36.281953 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-ce50ae4c-b4e6-4098-aa9f-3b462e3af938 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] [instance: 5bdc36c3-1db5-4ff2-808d-1295f314d522] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:44:36.290438 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ce50ae4c-b4e6-4098-aa9f-3b462e3af938 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:36.291386 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ce50ae4c-b4e6-4098-aa9f-3b462e3af938 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:36.292168 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ce50ae4c-b4e6-4098-aa9f-3b462e3af938 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:36.293480 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ce50ae4c-b4e6-4098-aa9f-3b462e3af938 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:36.294299 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ce50ae4c-b4e6-4098-aa9f-3b462e3af938 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:36.295341 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ce50ae4c-b4e6-4098-aa9f-3b462e3af938 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:36.322181 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.wsgi [None req-ce50ae4c-b4e6-4098-aa9f-3b462e3af938 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] HTTP exception thrown: Instance 5bdc36c3-1db5-4ff2-808d-1295f314d522 could not be found. Oct 19 00:44:36.325956 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-ce50ae4c-b4e6-4098-aa9f-3b462e3af938 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Request method failure captured: Oct 19 00:44:36.325956 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: request: GET /compute/v2.1/servers/5bdc36c3-1db5-4ff2-808d-1295f314d522 HTTP/1.1 Oct 19 00:44:36.325956 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Accept: application/json Oct 19 00:44:36.325956 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Accept-Encoding: identity Oct 19 00:44:36.325956 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Connection: close Oct 19 00:44:36.325956 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Content-Length: 0 Oct 19 00:44:36.325956 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Content-Type: application/json Oct 19 00:44:36.325956 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Host: 10.222.0.26 Oct 19 00:44:36.325956 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Openstack-System-Scope: None Oct 19 00:44:36.325956 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: User-Agent: python-urllib3/1.26.12 Oct 19 00:44:36.325956 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Auth-Token: gAAAAABjT0gKnENal7-Rh73WviQBQSLL5676mOhmhgH_YQb-aYaRiQLWVmjdDva3tE-fi-9DMsXk14oSgyRxVtRsla_-rGKcqb59timblqX-EY620cGLwQo3rRgl2mWDPAr19L4z-8pjV12JYT9LTCQHYeyjGdQSstwReJL2x7Na4aYUu9YdbF0 Oct 19 00:44:36.325956 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Domain-Id: None Oct 19 00:44:36.325956 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Domain-Name: None Oct 19 00:44:36.325956 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Identity-Status: Confirmed Oct 19 00:44:36.325956 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Is-Admin-Project: True Oct 19 00:44:36.325956 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Project-Domain-Id: default Oct 19 00:44:36.325956 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Project-Domain-Name: Default Oct 19 00:44:36.325956 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Project-Id: 1235d8b054b3405595fca761f08b73ae Oct 19 00:44:36.325956 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Project-Name: tempest-VolumesSnapshotTestJSON-2142930389 Oct 19 00:44:36.325956 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Role: member,reader Oct 19 00:44:36.325956 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Roles: member,reader Oct 19 00:44:36.328572 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/1235d8b054b3405595fca761f08b73ae"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/1235d8b054b3405595fca761f08b73ae"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/1235d8b054b3405595fca761f08b73ae"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_1235d8b054b3405595fca761f08b73ae"}]}] Oct 19 00:44:36.328572 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Tenant: tempest-VolumesSnapshotTestJSON-2142930389 Oct 19 00:44:36.328572 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Tenant-Id: 1235d8b054b3405595fca761f08b73ae Oct 19 00:44:36.328572 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Tenant-Name: tempest-VolumesSnapshotTestJSON-2142930389 Oct 19 00:44:36.328572 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User: tempest-VolumesSnapshotTestJSON-2142930389-project Oct 19 00:44:36.328572 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User-Domain-Id: default Oct 19 00:44:36.328572 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User-Domain-Name: Default Oct 19 00:44:36.328572 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User-Id: 39de6a6fff7c4cc08cc62aa86182d71d Oct 19 00:44:36.328572 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User-Name: tempest-VolumesSnapshotTestJSON-2142930389-project Oct 19 00:44:36.328572 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: method: > Oct 19 00:44:36.328572 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: exception: Instance 5bdc36c3-1db5-4ff2-808d-1295f314d522 could not be found. Oct 19 00:44:36.328572 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: action_args: {'id': '5bdc36c3-1db5-4ff2-808d-1295f314d522'} Oct 19 00:44:36.331071 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 19 00:44:36.331071 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 19 00:44:36.331071 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 19 00:44:36.331071 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 19 00:44:36.331071 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 19 00:44:36.331071 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 19 00:44:36.331071 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 19 00:44:36.331071 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 19 00:44:36.331071 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 19 00:44:36.331071 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 19 00:44:36.331071 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 5bdc36c3-1db5-4ff2-808d-1295f314d522 could not be found. Oct 19 00:44:36.331071 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:44:36.331071 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 19 00:44:36.331071 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:44:36.331071 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 19 00:44:36.331071 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 19 00:44:36.331071 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 19 00:44:36.331071 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 19 00:44:36.331071 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 19 00:44:36.331071 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 19 00:44:36.336439 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 19 00:44:36.336439 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 19 00:44:36.336439 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 19 00:44:36.336439 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 19 00:44:36.336439 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 19 00:44:36.336439 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 19 00:44:36.336439 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 19 00:44:36.336439 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 5bdc36c3-1db5-4ff2-808d-1295f314d522 could not be found. Oct 19 00:44:36.336439 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:44:36.336439 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 19 00:44:36.336439 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:44:36.336439 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 19 00:44:36.336439 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 19 00:44:36.336439 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 19 00:44:36.336439 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 19 00:44:36.336439 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 19 00:44:36.336439 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 5bdc36c3-1db5-4ff2-808d-1295f314d522 could not be found. Oct 19 00:44:36.336439 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:44:36.340840 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-ce50ae4c-b4e6-4098-aa9f-3b462e3af938 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Returning 404 to user: Instance 5bdc36c3-1db5-4ff2-808d-1295f314d522 could not be found. {{(pid=125641) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 19 00:44:36.340840 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-ce50ae4c-b4e6-4098-aa9f-3b462e3af938 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] 10.222.0.26 "GET /compute/v2.1/servers/5bdc36c3-1db5-4ff2-808d-1295f314d522" status: 404 len: 111 microversion: 2.1 time: 0.054245 Oct 19 00:44:36.340840 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 357/472] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:44:36 2022] GET /compute/v2.1/servers/5bdc36c3-1db5-4ff2-808d-1295f314d522 => generated 111 bytes in 56 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 19 00:44:36.340840 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-2b0c4ea1-e1a0-425c-a6fc-bb9e98c9cea2 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:44:36.340840 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-2b0c4ea1-e1a0-425c-a6fc-bb9e98c9cea2 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] [instance: d8ff0e88-597f-4db1-b5f4-852dd33c566d] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:44:36.354035 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-2b0c4ea1-e1a0-425c-a6fc-bb9e98c9cea2 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:36.356210 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-2b0c4ea1-e1a0-425c-a6fc-bb9e98c9cea2 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:36.357263 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-2b0c4ea1-e1a0-425c-a6fc-bb9e98c9cea2 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:36.358791 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-2b0c4ea1-e1a0-425c-a6fc-bb9e98c9cea2 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:36.359892 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-2b0c4ea1-e1a0-425c-a6fc-bb9e98c9cea2 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:36.360940 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-2b0c4ea1-e1a0-425c-a6fc-bb9e98c9cea2 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:36.469020 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-2b0c4ea1-e1a0-425c-a6fc-bb9e98c9cea2 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f8725c4d05b4dd79b05b43fba15980b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0b35c2038f6b4d0a9012b004f1416c00', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:44:36.480529 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-2b0c4ea1-e1a0-425c-a6fc-bb9e98c9cea2 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] 10.222.0.26 "GET /compute/v2.1/servers/d8ff0e88-597f-4db1-b5f4-852dd33c566d" status: 200 len: 1294 microversion: 2.1 time: 0.149339 Oct 19 00:44:36.481449 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 358/473] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:44:36 2022] GET /compute/v2.1/servers/d8ff0e88-597f-4db1-b5f4-852dd33c566d => generated 1294 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:44:36.491089 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-aea42386-baab-4ffa-9b9e-7db78753742a tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:44:36.491745 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-aea42386-baab-4ffa-9b9e-7db78753742a tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] [instance: 0b7c19ee-6d89-4855-bf4b-d35b65bb170b] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:44:36.499254 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-aea42386-baab-4ffa-9b9e-7db78753742a tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:36.499771 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-aea42386-baab-4ffa-9b9e-7db78753742a tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:36.500218 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-aea42386-baab-4ffa-9b9e-7db78753742a tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:36.500965 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-aea42386-baab-4ffa-9b9e-7db78753742a tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:36.501434 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-aea42386-baab-4ffa-9b9e-7db78753742a tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:36.501861 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-aea42386-baab-4ffa-9b9e-7db78753742a tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:36.753836 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-aea42386-baab-4ffa-9b9e-7db78753742a tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f738e67d8c2340028da1254b60d7810e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8684504b0eff415b9063cebae21ca667', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:44:36.774108 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-aea42386-baab-4ffa-9b9e-7db78753742a tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] 10.222.0.26 "GET /compute/v2.1/servers/0b7c19ee-6d89-4855-bf4b-d35b65bb170b" status: 200 len: 1266 microversion: 2.1 time: 0.290970 Oct 19 00:44:36.775697 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 359/474] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:44:36 2022] GET /compute/v2.1/servers/0b7c19ee-6d89-4855-bf4b-d35b65bb170b => generated 1266 bytes in 294 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:44:37.281680 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [req-dd63b1b4-01e3-41db-83b0-e2d7e1d09038 req-6067be2a-0657-47a3-bff7-4b170eae045d service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "0b7c19ee-6d89-4855-bf4b-d35b65bb170b", "tag": "3f92cbb3-bb62-457d-b0d7-92947884c5fb"}]} {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 19 00:44:37.297906 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-dd63b1b4-01e3-41db-83b0-e2d7e1d09038 req-6067be2a-0657-47a3-bff7-4b170eae045d service nova] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:37.298424 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-dd63b1b4-01e3-41db-83b0-e2d7e1d09038 req-6067be2a-0657-47a3-bff7-4b170eae045d service nova] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:37.299256 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-dd63b1b4-01e3-41db-83b0-e2d7e1d09038 req-6067be2a-0657-47a3-bff7-4b170eae045d service nova] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:37.320474 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG dbcounter [-] [125641] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=125641) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 00:44:37.324464 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.compute.server_external_events [req-dd63b1b4-01e3-41db-83b0-e2d7e1d09038 req-6067be2a-0657-47a3-bff7-4b170eae045d service nova] Creating event network-changed:3f92cbb3-bb62-457d-b0d7-92947884c5fb for instance 0b7c19ee-6d89-4855-bf4b-d35b65bb170b on hv-cinder-maste Oct 19 00:44:37.331840 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [req-dd63b1b4-01e3-41db-83b0-e2d7e1d09038 req-6067be2a-0657-47a3-bff7-4b170eae045d service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.054109 Oct 19 00:44:37.332730 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 360/475] 10.222.0.26 () {64 vars in 1317 bytes} [Wed Oct 19 00:44:37 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 19 00:44:37.341901 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-a0f79ccf-e872-4edd-8c18-559a76e0707b tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:44:37.342746 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-a0f79ccf-e872-4edd-8c18-559a76e0707b tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] [instance: f6051269-1d16-42db-8323-67a532dbcf40] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:44:37.352243 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-a0f79ccf-e872-4edd-8c18-559a76e0707b tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:37.352977 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-a0f79ccf-e872-4edd-8c18-559a76e0707b tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:37.354160 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-a0f79ccf-e872-4edd-8c18-559a76e0707b tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:37.355174 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-a0f79ccf-e872-4edd-8c18-559a76e0707b tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:37.355701 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-a0f79ccf-e872-4edd-8c18-559a76e0707b tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:37.356309 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-a0f79ccf-e872-4edd-8c18-559a76e0707b tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:37.513291 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-a0f79ccf-e872-4edd-8c18-559a76e0707b tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd3f0d615187d4779bdd7da8c9f24c796', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0151d58abd047a692f5cc4e7d05bea5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:44:37.529154 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-a0f79ccf-e872-4edd-8c18-559a76e0707b tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] 10.222.0.26 "GET /compute/v2.1/servers/f6051269-1d16-42db-8323-67a532dbcf40" status: 200 len: 1737 microversion: 2.1 time: 0.194479 Oct 19 00:44:37.530221 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 361/476] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:44:37 2022] GET /compute/v2.1/servers/f6051269-1d16-42db-8323-67a532dbcf40 => generated 1737 bytes in 197 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:44:37.539012 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-5da1358d-209f-4bd7-9037-e619c6be62e8 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:44:37.540148 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-5da1358d-209f-4bd7-9037-e619c6be62e8 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] [instance: d8ff0e88-597f-4db1-b5f4-852dd33c566d] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:44:37.551878 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-5da1358d-209f-4bd7-9037-e619c6be62e8 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:37.552740 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-5da1358d-209f-4bd7-9037-e619c6be62e8 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:37.553514 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-5da1358d-209f-4bd7-9037-e619c6be62e8 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:37.554871 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-5da1358d-209f-4bd7-9037-e619c6be62e8 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:37.555730 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-5da1358d-209f-4bd7-9037-e619c6be62e8 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:37.556570 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-5da1358d-209f-4bd7-9037-e619c6be62e8 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:37.630145 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-5da1358d-209f-4bd7-9037-e619c6be62e8 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f8725c4d05b4dd79b05b43fba15980b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0b35c2038f6b4d0a9012b004f1416c00', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:44:37.639155 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-5da1358d-209f-4bd7-9037-e619c6be62e8 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] 10.222.0.26 "GET /compute/v2.1/servers/d8ff0e88-597f-4db1-b5f4-852dd33c566d" status: 200 len: 1300 microversion: 2.1 time: 0.107387 Oct 19 00:44:37.639765 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 362/477] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:44:37 2022] GET /compute/v2.1/servers/d8ff0e88-597f-4db1-b5f4-852dd33c566d => generated 1300 bytes in 109 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:44:37.801087 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-26cc3d27-6924-4451-97cd-be790795f450 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:44:37.801715 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-26cc3d27-6924-4451-97cd-be790795f450 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] [instance: 0b7c19ee-6d89-4855-bf4b-d35b65bb170b] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:44:37.808778 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-26cc3d27-6924-4451-97cd-be790795f450 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:37.809273 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-26cc3d27-6924-4451-97cd-be790795f450 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:37.809716 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-26cc3d27-6924-4451-97cd-be790795f450 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:37.810446 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-26cc3d27-6924-4451-97cd-be790795f450 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:37.810946 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-26cc3d27-6924-4451-97cd-be790795f450 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:37.811379 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-26cc3d27-6924-4451-97cd-be790795f450 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:37.962669 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-26cc3d27-6924-4451-97cd-be790795f450 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f738e67d8c2340028da1254b60d7810e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8684504b0eff415b9063cebae21ca667', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:44:37.972445 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-26cc3d27-6924-4451-97cd-be790795f450 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] 10.222.0.26 "GET /compute/v2.1/servers/0b7c19ee-6d89-4855-bf4b-d35b65bb170b" status: 200 len: 1266 microversion: 2.1 time: 0.175320 Oct 19 00:44:37.973107 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 363/478] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:44:37 2022] GET /compute/v2.1/servers/0b7c19ee-6d89-4855-bf4b-d35b65bb170b => generated 1266 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 19 00:44:38.555768 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-28a259f4-cc2b-43a0-bff9-6a16247d0288 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:44:38.557350 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-28a259f4-cc2b-43a0-bff9-6a16247d0288 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] [instance: f6051269-1d16-42db-8323-67a532dbcf40] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:44:38.564991 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-28a259f4-cc2b-43a0-bff9-6a16247d0288 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:38.565485 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-28a259f4-cc2b-43a0-bff9-6a16247d0288 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:38.565958 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-28a259f4-cc2b-43a0-bff9-6a16247d0288 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:38.566738 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-28a259f4-cc2b-43a0-bff9-6a16247d0288 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:38.567197 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-28a259f4-cc2b-43a0-bff9-6a16247d0288 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:38.567626 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-28a259f4-cc2b-43a0-bff9-6a16247d0288 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:38.717940 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-28a259f4-cc2b-43a0-bff9-6a16247d0288 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd3f0d615187d4779bdd7da8c9f24c796', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0151d58abd047a692f5cc4e7d05bea5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:44:38.727511 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-28a259f4-cc2b-43a0-bff9-6a16247d0288 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] 10.222.0.26 "GET /compute/v2.1/servers/f6051269-1d16-42db-8323-67a532dbcf40" status: 200 len: 1737 microversion: 2.1 time: 0.176099 Oct 19 00:44:38.728133 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 364/479] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:44:38 2022] GET /compute/v2.1/servers/f6051269-1d16-42db-8323-67a532dbcf40 => generated 1737 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 19 00:44:38.733739 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-206740d6-9721-463c-8514-865202fd0c79 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:44:38.734474 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-206740d6-9721-463c-8514-865202fd0c79 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] [instance: d8ff0e88-597f-4db1-b5f4-852dd33c566d] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:44:38.742397 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-206740d6-9721-463c-8514-865202fd0c79 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:38.743055 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-206740d6-9721-463c-8514-865202fd0c79 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:38.743490 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-206740d6-9721-463c-8514-865202fd0c79 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:38.744842 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-206740d6-9721-463c-8514-865202fd0c79 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:38.745382 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-206740d6-9721-463c-8514-865202fd0c79 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:38.745864 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-206740d6-9721-463c-8514-865202fd0c79 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:38.819035 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-206740d6-9721-463c-8514-865202fd0c79 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f8725c4d05b4dd79b05b43fba15980b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0b35c2038f6b4d0a9012b004f1416c00', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:44:38.827895 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-206740d6-9721-463c-8514-865202fd0c79 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] 10.222.0.26 "GET /compute/v2.1/servers/d8ff0e88-597f-4db1-b5f4-852dd33c566d" status: 200 len: 1300 microversion: 2.1 time: 0.098117 Oct 19 00:44:38.828497 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 365/480] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:44:38 2022] GET /compute/v2.1/servers/d8ff0e88-597f-4db1-b5f4-852dd33c566d => generated 1300 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:44:38.996495 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-2d294ac4-482d-4ca8-954b-2b2248613790 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:44:38.997115 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-2d294ac4-482d-4ca8-954b-2b2248613790 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] [instance: 0b7c19ee-6d89-4855-bf4b-d35b65bb170b] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:44:39.003429 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-2d294ac4-482d-4ca8-954b-2b2248613790 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:39.003923 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-2d294ac4-482d-4ca8-954b-2b2248613790 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:39.004361 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-2d294ac4-482d-4ca8-954b-2b2248613790 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:39.005073 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-2d294ac4-482d-4ca8-954b-2b2248613790 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:39.005535 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-2d294ac4-482d-4ca8-954b-2b2248613790 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:39.005958 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-2d294ac4-482d-4ca8-954b-2b2248613790 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:39.151657 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-2d294ac4-482d-4ca8-954b-2b2248613790 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f738e67d8c2340028da1254b60d7810e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8684504b0eff415b9063cebae21ca667', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:44:39.159872 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-2d294ac4-482d-4ca8-954b-2b2248613790 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] 10.222.0.26 "GET /compute/v2.1/servers/0b7c19ee-6d89-4855-bf4b-d35b65bb170b" status: 200 len: 1266 microversion: 2.1 time: 0.167532 Oct 19 00:44:39.160465 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 366/481] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:44:38 2022] GET /compute/v2.1/servers/0b7c19ee-6d89-4855-bf4b-d35b65bb170b => generated 1266 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 19 00:44:39.754698 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-ce590ab7-6146-467b-8f79-71e4731594f2 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:44:39.755725 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-ce590ab7-6146-467b-8f79-71e4731594f2 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] [instance: f6051269-1d16-42db-8323-67a532dbcf40] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:44:39.763616 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ce590ab7-6146-467b-8f79-71e4731594f2 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:39.764152 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ce590ab7-6146-467b-8f79-71e4731594f2 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:39.764597 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ce590ab7-6146-467b-8f79-71e4731594f2 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:39.765311 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ce590ab7-6146-467b-8f79-71e4731594f2 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:39.765815 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ce590ab7-6146-467b-8f79-71e4731594f2 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:39.766244 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ce590ab7-6146-467b-8f79-71e4731594f2 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:40.010429 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-ce590ab7-6146-467b-8f79-71e4731594f2 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd3f0d615187d4779bdd7da8c9f24c796', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0151d58abd047a692f5cc4e7d05bea5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:44:40.027843 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-ce590ab7-6146-467b-8f79-71e4731594f2 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] 10.222.0.26 "GET /compute/v2.1/servers/f6051269-1d16-42db-8323-67a532dbcf40" status: 200 len: 1737 microversion: 2.1 time: 0.277062 Oct 19 00:44:40.029394 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 367/482] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:44:39 2022] GET /compute/v2.1/servers/f6051269-1d16-42db-8323-67a532dbcf40 => generated 1737 bytes in 279 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 19 00:44:40.038795 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-9a5aaba3-541e-40ac-a9f6-bc3f34ad713a tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:44:40.040285 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-9a5aaba3-541e-40ac-a9f6-bc3f34ad713a tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] [instance: d8ff0e88-597f-4db1-b5f4-852dd33c566d] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:44:40.060227 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-9a5aaba3-541e-40ac-a9f6-bc3f34ad713a tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:40.061152 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-9a5aaba3-541e-40ac-a9f6-bc3f34ad713a tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:40.061937 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-9a5aaba3-541e-40ac-a9f6-bc3f34ad713a tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:40.064206 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-9a5aaba3-541e-40ac-a9f6-bc3f34ad713a tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:40.065067 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-9a5aaba3-541e-40ac-a9f6-bc3f34ad713a tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:40.065842 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-9a5aaba3-541e-40ac-a9f6-bc3f34ad713a tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:40.182788 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-9a5aaba3-541e-40ac-a9f6-bc3f34ad713a tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f8725c4d05b4dd79b05b43fba15980b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0b35c2038f6b4d0a9012b004f1416c00', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:44:40.196458 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-9a5aaba3-541e-40ac-a9f6-bc3f34ad713a tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] 10.222.0.26 "GET /compute/v2.1/servers/d8ff0e88-597f-4db1-b5f4-852dd33c566d" status: 200 len: 1300 microversion: 2.1 time: 0.165155 Oct 19 00:44:40.198327 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 368/483] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:44:40 2022] GET /compute/v2.1/servers/d8ff0e88-597f-4db1-b5f4-852dd33c566d => generated 1300 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:44:40.215262 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-fa360842-c12a-401b-8738-157b5ac0bc27 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:44:40.216146 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-fa360842-c12a-401b-8738-157b5ac0bc27 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] [instance: 0b7c19ee-6d89-4855-bf4b-d35b65bb170b] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:44:40.226221 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-fa360842-c12a-401b-8738-157b5ac0bc27 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:40.227328 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-fa360842-c12a-401b-8738-157b5ac0bc27 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:40.227920 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-fa360842-c12a-401b-8738-157b5ac0bc27 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:40.229870 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-fa360842-c12a-401b-8738-157b5ac0bc27 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:40.230300 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-fa360842-c12a-401b-8738-157b5ac0bc27 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:40.231586 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-fa360842-c12a-401b-8738-157b5ac0bc27 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:40.392603 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-fa360842-c12a-401b-8738-157b5ac0bc27 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f738e67d8c2340028da1254b60d7810e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8684504b0eff415b9063cebae21ca667', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:44:40.401033 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-fa360842-c12a-401b-8738-157b5ac0bc27 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] 10.222.0.26 "GET /compute/v2.1/servers/0b7c19ee-6d89-4855-bf4b-d35b65bb170b" status: 200 len: 1266 microversion: 2.1 time: 0.201816 Oct 19 00:44:40.401977 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 369/484] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:44:40 2022] GET /compute/v2.1/servers/0b7c19ee-6d89-4855-bf4b-d35b65bb170b => generated 1266 bytes in 203 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:44:41.065674 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-473a0057-afef-43cd-8254-7524636fd126 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:44:41.066819 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-473a0057-afef-43cd-8254-7524636fd126 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] [instance: f6051269-1d16-42db-8323-67a532dbcf40] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:44:41.076402 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-473a0057-afef-43cd-8254-7524636fd126 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:41.077255 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-473a0057-afef-43cd-8254-7524636fd126 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:41.077835 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-473a0057-afef-43cd-8254-7524636fd126 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:41.078888 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-473a0057-afef-43cd-8254-7524636fd126 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:41.079337 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-473a0057-afef-43cd-8254-7524636fd126 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:41.080027 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-473a0057-afef-43cd-8254-7524636fd126 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:41.231334 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-473a0057-afef-43cd-8254-7524636fd126 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd3f0d615187d4779bdd7da8c9f24c796', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0151d58abd047a692f5cc4e7d05bea5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:44:41.240747 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-473a0057-afef-43cd-8254-7524636fd126 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] 10.222.0.26 "GET /compute/v2.1/servers/f6051269-1d16-42db-8323-67a532dbcf40" status: 200 len: 1737 microversion: 2.1 time: 0.179981 Oct 19 00:44:41.251428 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 370/485] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:44:41 2022] GET /compute/v2.1/servers/f6051269-1d16-42db-8323-67a532dbcf40 => generated 1737 bytes in 191 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 19 00:44:41.266094 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-2510d9c6-2ddb-42de-b7bf-97b47595c6e3 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:44:41.273149 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-2510d9c6-2ddb-42de-b7bf-97b47595c6e3 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] [instance: d8ff0e88-597f-4db1-b5f4-852dd33c566d] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:44:41.277076 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-2510d9c6-2ddb-42de-b7bf-97b47595c6e3 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:41.277076 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-2510d9c6-2ddb-42de-b7bf-97b47595c6e3 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:41.277684 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-2510d9c6-2ddb-42de-b7bf-97b47595c6e3 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:41.278521 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-2510d9c6-2ddb-42de-b7bf-97b47595c6e3 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:41.279783 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-2510d9c6-2ddb-42de-b7bf-97b47595c6e3 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:41.280231 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-2510d9c6-2ddb-42de-b7bf-97b47595c6e3 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:41.502527 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-2510d9c6-2ddb-42de-b7bf-97b47595c6e3 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f8725c4d05b4dd79b05b43fba15980b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0b35c2038f6b4d0a9012b004f1416c00', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:44:41.516172 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-2510d9c6-2ddb-42de-b7bf-97b47595c6e3 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] 10.222.0.26 "GET /compute/v2.1/servers/d8ff0e88-597f-4db1-b5f4-852dd33c566d" status: 200 len: 1367 microversion: 2.1 time: 0.262984 Oct 19 00:44:41.518331 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 371/486] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:44:41 2022] GET /compute/v2.1/servers/d8ff0e88-597f-4db1-b5f4-852dd33c566d => generated 1367 bytes in 266 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:44:41.535297 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-a62680d5-bab9-4eaf-b225-67c0780b3163 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:44:41.537614 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-a62680d5-bab9-4eaf-b225-67c0780b3163 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] [instance: 0b7c19ee-6d89-4855-bf4b-d35b65bb170b] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:44:41.577325 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-a62680d5-bab9-4eaf-b225-67c0780b3163 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:41.578699 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-a62680d5-bab9-4eaf-b225-67c0780b3163 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:41.579963 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-a62680d5-bab9-4eaf-b225-67c0780b3163 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:41.581543 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-a62680d5-bab9-4eaf-b225-67c0780b3163 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:41.583283 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-a62680d5-bab9-4eaf-b225-67c0780b3163 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:41.584380 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-a62680d5-bab9-4eaf-b225-67c0780b3163 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:41.829973 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-a62680d5-bab9-4eaf-b225-67c0780b3163 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f738e67d8c2340028da1254b60d7810e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8684504b0eff415b9063cebae21ca667', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:44:41.846527 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-a62680d5-bab9-4eaf-b225-67c0780b3163 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] 10.222.0.26 "GET /compute/v2.1/servers/0b7c19ee-6d89-4855-bf4b-d35b65bb170b" status: 200 len: 1266 microversion: 2.1 time: 0.325864 Oct 19 00:44:41.847713 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 372/487] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:44:41 2022] GET /compute/v2.1/servers/0b7c19ee-6d89-4855-bf4b-d35b65bb170b => generated 1266 bytes in 329 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:44:42.269351 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-4d73872c-a6e2-4b70-b635-1d2f9397a6a0 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:44:42.270833 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-4d73872c-a6e2-4b70-b635-1d2f9397a6a0 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] [instance: f6051269-1d16-42db-8323-67a532dbcf40] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:44:42.280427 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-4d73872c-a6e2-4b70-b635-1d2f9397a6a0 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:42.281277 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-4d73872c-a6e2-4b70-b635-1d2f9397a6a0 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:42.281678 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-4d73872c-a6e2-4b70-b635-1d2f9397a6a0 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:42.282779 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-4d73872c-a6e2-4b70-b635-1d2f9397a6a0 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:42.283469 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-4d73872c-a6e2-4b70-b635-1d2f9397a6a0 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:42.284031 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-4d73872c-a6e2-4b70-b635-1d2f9397a6a0 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:42.758476 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-4d73872c-a6e2-4b70-b635-1d2f9397a6a0 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd3f0d615187d4779bdd7da8c9f24c796', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0151d58abd047a692f5cc4e7d05bea5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:44:42.768317 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-4d73872c-a6e2-4b70-b635-1d2f9397a6a0 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] 10.222.0.26 "GET /compute/v2.1/servers/f6051269-1d16-42db-8323-67a532dbcf40" status: 200 len: 1737 microversion: 2.1 time: 0.503299 Oct 19 00:44:42.768924 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 373/488] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:44:42 2022] GET /compute/v2.1/servers/f6051269-1d16-42db-8323-67a532dbcf40 => generated 1737 bytes in 505 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:44:42.773904 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-c0026aec-26b2-43ba-9dbd-97a8cf117945 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:44:42.774554 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-c0026aec-26b2-43ba-9dbd-97a8cf117945 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] [instance: d8ff0e88-597f-4db1-b5f4-852dd33c566d] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:44:42.782416 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-c0026aec-26b2-43ba-9dbd-97a8cf117945 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:42.783084 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-c0026aec-26b2-43ba-9dbd-97a8cf117945 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:42.783651 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-c0026aec-26b2-43ba-9dbd-97a8cf117945 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:42.784532 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-c0026aec-26b2-43ba-9dbd-97a8cf117945 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:42.785091 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-c0026aec-26b2-43ba-9dbd-97a8cf117945 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:42.785698 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-c0026aec-26b2-43ba-9dbd-97a8cf117945 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:42.939402 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-c0026aec-26b2-43ba-9dbd-97a8cf117945 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f8725c4d05b4dd79b05b43fba15980b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0b35c2038f6b4d0a9012b004f1416c00', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:44:42.948998 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-c0026aec-26b2-43ba-9dbd-97a8cf117945 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] 10.222.0.26 "GET /compute/v2.1/servers/d8ff0e88-597f-4db1-b5f4-852dd33c566d" status: 200 len: 1367 microversion: 2.1 time: 0.178958 Oct 19 00:44:42.949930 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 374/489] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:44:42 2022] GET /compute/v2.1/servers/d8ff0e88-597f-4db1-b5f4-852dd33c566d => generated 1367 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:44:42.954750 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [req-ee45477f-68db-4c1e-ae59-0dc1dc588243 req-d93cb58a-fd90-4cad-8303-f58ddb093763 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "d8ff0e88-597f-4db1-b5f4-852dd33c566d", "tag": "1598d129-803e-437a-be2e-c9127e38bd88"}]} {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 19 00:44:42.966291 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-ee45477f-68db-4c1e-ae59-0dc1dc588243 req-d93cb58a-fd90-4cad-8303-f58ddb093763 service nova] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:42.966819 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-ee45477f-68db-4c1e-ae59-0dc1dc588243 req-d93cb58a-fd90-4cad-8303-f58ddb093763 service nova] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:42.967261 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-ee45477f-68db-4c1e-ae59-0dc1dc588243 req-d93cb58a-fd90-4cad-8303-f58ddb093763 service nova] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:42.980254 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.compute.server_external_events [req-ee45477f-68db-4c1e-ae59-0dc1dc588243 req-d93cb58a-fd90-4cad-8303-f58ddb093763 service nova] Creating event network-changed:1598d129-803e-437a-be2e-c9127e38bd88 for instance d8ff0e88-597f-4db1-b5f4-852dd33c566d on hv-cinder-maste Oct 19 00:44:42.985547 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [req-ee45477f-68db-4c1e-ae59-0dc1dc588243 req-d93cb58a-fd90-4cad-8303-f58ddb093763 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.034546 Oct 19 00:44:42.986252 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 375/490] 10.222.0.26 () {64 vars in 1317 bytes} [Wed Oct 19 00:44:42 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 19 00:44:42.992680 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-d7c7c8d2-0303-43e4-af67-bef3ac1704a2 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:44:42.993394 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-d7c7c8d2-0303-43e4-af67-bef3ac1704a2 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] [instance: 0b7c19ee-6d89-4855-bf4b-d35b65bb170b] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:44:43.000270 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d7c7c8d2-0303-43e4-af67-bef3ac1704a2 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:43.000778 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d7c7c8d2-0303-43e4-af67-bef3ac1704a2 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:43.001211 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d7c7c8d2-0303-43e4-af67-bef3ac1704a2 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:43.001933 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d7c7c8d2-0303-43e4-af67-bef3ac1704a2 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:43.002396 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d7c7c8d2-0303-43e4-af67-bef3ac1704a2 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:43.002857 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d7c7c8d2-0303-43e4-af67-bef3ac1704a2 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:43.160852 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-d7c7c8d2-0303-43e4-af67-bef3ac1704a2 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f738e67d8c2340028da1254b60d7810e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8684504b0eff415b9063cebae21ca667', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:44:43.173928 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-d7c7c8d2-0303-43e4-af67-bef3ac1704a2 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] 10.222.0.26 "GET /compute/v2.1/servers/0b7c19ee-6d89-4855-bf4b-d35b65bb170b" status: 200 len: 1266 microversion: 2.1 time: 0.185764 Oct 19 00:44:43.175026 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 376/491] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:44:42 2022] GET /compute/v2.1/servers/0b7c19ee-6d89-4855-bf4b-d35b65bb170b => generated 1266 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:44:43.801938 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-938a9a45-443c-48ef-9ae4-712c7ab33c2e tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:44:43.803096 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-938a9a45-443c-48ef-9ae4-712c7ab33c2e tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] [instance: f6051269-1d16-42db-8323-67a532dbcf40] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:44:43.816548 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-938a9a45-443c-48ef-9ae4-712c7ab33c2e tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:43.817289 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-938a9a45-443c-48ef-9ae4-712c7ab33c2e tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:43.818066 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-938a9a45-443c-48ef-9ae4-712c7ab33c2e tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:43.819343 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-938a9a45-443c-48ef-9ae4-712c7ab33c2e tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:43.820157 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-938a9a45-443c-48ef-9ae4-712c7ab33c2e tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:43.820971 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-938a9a45-443c-48ef-9ae4-712c7ab33c2e tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:43.985984 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-938a9a45-443c-48ef-9ae4-712c7ab33c2e tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd3f0d615187d4779bdd7da8c9f24c796', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0151d58abd047a692f5cc4e7d05bea5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:44:43.997464 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-938a9a45-443c-48ef-9ae4-712c7ab33c2e tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] 10.222.0.26 "GET /compute/v2.1/servers/f6051269-1d16-42db-8323-67a532dbcf40" status: 200 len: 1737 microversion: 2.1 time: 0.203472 Oct 19 00:44:43.998375 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 377/492] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:44:43 2022] GET /compute/v2.1/servers/f6051269-1d16-42db-8323-67a532dbcf40 => generated 1737 bytes in 205 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:44:44.003978 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-babea24f-e700-47a6-af24-0d9b6e95f4da tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:44:44.004793 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-babea24f-e700-47a6-af24-0d9b6e95f4da tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] [instance: d8ff0e88-597f-4db1-b5f4-852dd33c566d] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:44:44.014423 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-babea24f-e700-47a6-af24-0d9b6e95f4da tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:44.015057 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-babea24f-e700-47a6-af24-0d9b6e95f4da tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:44.015713 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-babea24f-e700-47a6-af24-0d9b6e95f4da tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:44.016587 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-babea24f-e700-47a6-af24-0d9b6e95f4da tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:44.017147 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-babea24f-e700-47a6-af24-0d9b6e95f4da tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:44.017819 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-babea24f-e700-47a6-af24-0d9b6e95f4da tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:44.022847 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG dbcounter [-] [125641] Writing DB stats nova_cell1:SELECT=100 {{(pid=125641) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 00:44:44.171347 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-babea24f-e700-47a6-af24-0d9b6e95f4da tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f8725c4d05b4dd79b05b43fba15980b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0b35c2038f6b4d0a9012b004f1416c00', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:44:44.180003 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-babea24f-e700-47a6-af24-0d9b6e95f4da tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] 10.222.0.26 "GET /compute/v2.1/servers/d8ff0e88-597f-4db1-b5f4-852dd33c566d" status: 200 len: 1367 microversion: 2.1 time: 0.180358 Oct 19 00:44:44.180679 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 378/493] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:44:43 2022] GET /compute/v2.1/servers/d8ff0e88-597f-4db1-b5f4-852dd33c566d => generated 1367 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:44:44.211383 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-a06fec12-774e-42d6-ae47-87a8ed75f40f tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:44:44.212048 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-a06fec12-774e-42d6-ae47-87a8ed75f40f tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] [instance: 0b7c19ee-6d89-4855-bf4b-d35b65bb170b] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:44:44.219990 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-a06fec12-774e-42d6-ae47-87a8ed75f40f tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:44.220494 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-a06fec12-774e-42d6-ae47-87a8ed75f40f tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:44.220927 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-a06fec12-774e-42d6-ae47-87a8ed75f40f tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:44.221648 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-a06fec12-774e-42d6-ae47-87a8ed75f40f tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:44.222100 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-a06fec12-774e-42d6-ae47-87a8ed75f40f tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:44.222553 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-a06fec12-774e-42d6-ae47-87a8ed75f40f tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:44.341130 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG dbcounter [-] [125641] Writing DB stats nova_cell0:SELECT=4 {{(pid=125641) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 00:44:44.375122 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-a06fec12-774e-42d6-ae47-87a8ed75f40f tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f738e67d8c2340028da1254b60d7810e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8684504b0eff415b9063cebae21ca667', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:44:44.383414 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-a06fec12-774e-42d6-ae47-87a8ed75f40f tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] 10.222.0.26 "GET /compute/v2.1/servers/0b7c19ee-6d89-4855-bf4b-d35b65bb170b" status: 200 len: 1266 microversion: 2.1 time: 0.176023 Oct 19 00:44:44.384008 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 379/494] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:44:44 2022] GET /compute/v2.1/servers/0b7c19ee-6d89-4855-bf4b-d35b65bb170b => generated 1266 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 19 00:44:44.885371 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-8909c1b8-1782-4b3f-9952-422222e5d6b7 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesSnapshotTestJSON-instance-845063150", "imageRef": "b387b4f8-f75e-4a68-b49c-a015bf61c46b", "flavorRef": "452", "networks": [{"uuid": "febcf320-b179-4c31-a898-24a66a89b934"}]}} {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 19 00:44:44.951617 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.network.neutron [None req-8909c1b8-1782-4b3f-9952-422222e5d6b7 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] validate_networks() for [('febcf320-b179-4c31-a898-24a66a89b934', None, None, None, None, None)] {{(pid=125641) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 19 00:44:45.284731 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG dbcounter [-] [125641] Writing DB stats nova_cell0:SELECT=18 {{(pid=125641) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 00:44:45.490455 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.virt.hardware [None req-8909c1b8-1782-4b3f-9952-422222e5d6b7 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Flavor limits 0:0:0 {{(pid=125641) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 19 00:44:45.490904 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.virt.hardware [None req-8909c1b8-1782-4b3f-9952-422222e5d6b7 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Image limits 0:0:0 {{(pid=125641) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 19 00:44:45.491383 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.virt.hardware [None req-8909c1b8-1782-4b3f-9952-422222e5d6b7 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Flavor pref 0:0:0 {{(pid=125641) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 19 00:44:45.491945 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.virt.hardware [None req-8909c1b8-1782-4b3f-9952-422222e5d6b7 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Image pref 0:0:0 {{(pid=125641) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 19 00:44:45.492366 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.virt.hardware [None req-8909c1b8-1782-4b3f-9952-422222e5d6b7 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=125641) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 19 00:44:45.505340 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-8909c1b8-1782-4b3f-9952-422222e5d6b7 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:45.506160 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-8909c1b8-1782-4b3f-9952-422222e5d6b7 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:45.506988 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-8909c1b8-1782-4b3f-9952-422222e5d6b7 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:45.508360 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-8909c1b8-1782-4b3f-9952-422222e5d6b7 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:45.509226 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-8909c1b8-1782-4b3f-9952-422222e5d6b7 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:45.510311 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-8909c1b8-1782-4b3f-9952-422222e5d6b7 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:45.537488 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.quota [None req-8909c1b8-1782-4b3f-9952-422222e5d6b7 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Getting quotas for project 1235d8b054b3405595fca761f08b73ae. Resources: {'cores', 'ram', 'instances'} {{(pid=125641) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 19 00:44:45.543099 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.quota [None req-8909c1b8-1782-4b3f-9952-422222e5d6b7 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Getting quotas for user 39de6a6fff7c4cc08cc62aa86182d71d and project 1235d8b054b3405595fca761f08b73ae. Resources: {'cores', 'ram', 'instances'} {{(pid=125641) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 19 00:44:45.557772 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-8909c1b8-1782-4b3f-9952-422222e5d6b7 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Going to run 1 instances... {{(pid=125641) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 19 00:44:45.559878 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-8909c1b8-1782-4b3f-9952-422222e5d6b7 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] [instance: c7dd981d-4382-47dc-8d9c-effae6f630f3] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='b387b4f8-f75e-4a68-b49c-a015bf61c46b',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=125641) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 19 00:44:45.615283 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-8909c1b8-1782-4b3f-9952-422222e5d6b7 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.734050 Oct 19 00:44:45.616369 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 380/495] 10.222.0.26 () {60 vars in 1147 bytes} [Wed Oct 19 00:44:44 2022] POST /compute/v2.1/servers => generated 380 bytes in 736 msecs (HTTP/1.1 202) 10 headers in 451 bytes (2 switches on core 0) Oct 19 00:44:45.621362 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-cf56088e-b791-41d3-aa5f-40a4c1c06143 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:44:45.622095 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-cf56088e-b791-41d3-aa5f-40a4c1c06143 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] [instance: f6051269-1d16-42db-8323-67a532dbcf40] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:44:45.629845 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-cf56088e-b791-41d3-aa5f-40a4c1c06143 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:45.630353 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-cf56088e-b791-41d3-aa5f-40a4c1c06143 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:45.630837 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-cf56088e-b791-41d3-aa5f-40a4c1c06143 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:45.631628 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-cf56088e-b791-41d3-aa5f-40a4c1c06143 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:45.632044 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-cf56088e-b791-41d3-aa5f-40a4c1c06143 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:45.632626 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-cf56088e-b791-41d3-aa5f-40a4c1c06143 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:45.846683 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-cf56088e-b791-41d3-aa5f-40a4c1c06143 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd3f0d615187d4779bdd7da8c9f24c796', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0151d58abd047a692f5cc4e7d05bea5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:44:45.857392 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-cf56088e-b791-41d3-aa5f-40a4c1c06143 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] 10.222.0.26 "GET /compute/v2.1/servers/f6051269-1d16-42db-8323-67a532dbcf40" status: 200 len: 1737 microversion: 2.1 time: 0.239935 Oct 19 00:44:45.858183 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 381/496] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:44:45 2022] GET /compute/v2.1/servers/f6051269-1d16-42db-8323-67a532dbcf40 => generated 1737 bytes in 241 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:44:45.867441 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-462e8b7e-3646-4fc3-9e5c-2b0208b09ab8 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:44:45.868493 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-462e8b7e-3646-4fc3-9e5c-2b0208b09ab8 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] [instance: d8ff0e88-597f-4db1-b5f4-852dd33c566d] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:44:45.882892 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-462e8b7e-3646-4fc3-9e5c-2b0208b09ab8 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:45.883932 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-462e8b7e-3646-4fc3-9e5c-2b0208b09ab8 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:45.885413 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-462e8b7e-3646-4fc3-9e5c-2b0208b09ab8 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:45.890699 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-462e8b7e-3646-4fc3-9e5c-2b0208b09ab8 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:45.891502 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-462e8b7e-3646-4fc3-9e5c-2b0208b09ab8 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:45.892081 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-462e8b7e-3646-4fc3-9e5c-2b0208b09ab8 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:46.140303 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-462e8b7e-3646-4fc3-9e5c-2b0208b09ab8 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f8725c4d05b4dd79b05b43fba15980b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0b35c2038f6b4d0a9012b004f1416c00', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:44:46.154856 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-462e8b7e-3646-4fc3-9e5c-2b0208b09ab8 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] 10.222.0.26 "GET /compute/v2.1/servers/d8ff0e88-597f-4db1-b5f4-852dd33c566d" status: 200 len: 1367 microversion: 2.1 time: 0.294802 Oct 19 00:44:46.155899 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 382/497] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:44:45 2022] GET /compute/v2.1/servers/d8ff0e88-597f-4db1-b5f4-852dd33c566d => generated 1367 bytes in 297 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:44:46.165504 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-808c8538-6237-42e5-8420-1a67546a58df tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:44:46.167238 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-808c8538-6237-42e5-8420-1a67546a58df tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] [instance: 0b7c19ee-6d89-4855-bf4b-d35b65bb170b] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:44:46.178530 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-808c8538-6237-42e5-8420-1a67546a58df tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:46.179706 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-808c8538-6237-42e5-8420-1a67546a58df tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:46.180547 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-808c8538-6237-42e5-8420-1a67546a58df tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:46.182427 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-808c8538-6237-42e5-8420-1a67546a58df tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:46.183309 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-808c8538-6237-42e5-8420-1a67546a58df tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:46.184066 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-808c8538-6237-42e5-8420-1a67546a58df tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:46.373396 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-808c8538-6237-42e5-8420-1a67546a58df tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f738e67d8c2340028da1254b60d7810e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8684504b0eff415b9063cebae21ca667', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:44:46.382444 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-808c8538-6237-42e5-8420-1a67546a58df tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] 10.222.0.26 "GET /compute/v2.1/servers/0b7c19ee-6d89-4855-bf4b-d35b65bb170b" status: 200 len: 1266 microversion: 2.1 time: 0.224958 Oct 19 00:44:46.382987 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 383/498] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:44:46 2022] GET /compute/v2.1/servers/0b7c19ee-6d89-4855-bf4b-d35b65bb170b => generated 1266 bytes in 227 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:44:46.388156 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-1b011fe3-e8a8-472a-a380-f5cacc979c7c tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:44:46.388777 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-1b011fe3-e8a8-472a-a380-f5cacc979c7c tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] [instance: c7dd981d-4382-47dc-8d9c-effae6f630f3] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:44:46.395402 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-1b011fe3-e8a8-472a-a380-f5cacc979c7c tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:46.395895 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-1b011fe3-e8a8-472a-a380-f5cacc979c7c tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:46.396332 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-1b011fe3-e8a8-472a-a380-f5cacc979c7c tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:46.397044 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-1b011fe3-e8a8-472a-a380-f5cacc979c7c tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:46.397500 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-1b011fe3-e8a8-472a-a380-f5cacc979c7c tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:46.397928 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-1b011fe3-e8a8-472a-a380-f5cacc979c7c tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:46.475539 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-1b011fe3-e8a8-472a-a380-f5cacc979c7c tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '39de6a6fff7c4cc08cc62aa86182d71d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1235d8b054b3405595fca761f08b73ae', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:44:46.484994 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-1b011fe3-e8a8-472a-a380-f5cacc979c7c tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] 10.222.0.26 "GET /compute/v2.1/servers/c7dd981d-4382-47dc-8d9c-effae6f630f3" status: 200 len: 1222 microversion: 2.1 time: 0.100914 Oct 19 00:44:46.485603 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 384/499] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:44:46 2022] GET /compute/v2.1/servers/c7dd981d-4382-47dc-8d9c-effae6f630f3 => generated 1222 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:44:46.898866 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-93c79fa3-e415-41eb-a8a5-0210184368dc tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:44:46.899563 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-93c79fa3-e415-41eb-a8a5-0210184368dc tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] [instance: f6051269-1d16-42db-8323-67a532dbcf40] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:44:46.907289 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-93c79fa3-e415-41eb-a8a5-0210184368dc tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:46.907779 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-93c79fa3-e415-41eb-a8a5-0210184368dc tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:46.908214 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-93c79fa3-e415-41eb-a8a5-0210184368dc tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:46.908926 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-93c79fa3-e415-41eb-a8a5-0210184368dc tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:46.909382 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-93c79fa3-e415-41eb-a8a5-0210184368dc tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:46.909811 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-93c79fa3-e415-41eb-a8a5-0210184368dc tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:47.071225 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-93c79fa3-e415-41eb-a8a5-0210184368dc tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd3f0d615187d4779bdd7da8c9f24c796', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0151d58abd047a692f5cc4e7d05bea5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:44:47.088664 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-93c79fa3-e415-41eb-a8a5-0210184368dc tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] 10.222.0.26 "GET /compute/v2.1/servers/f6051269-1d16-42db-8323-67a532dbcf40" status: 200 len: 1737 microversion: 2.1 time: 0.193304 Oct 19 00:44:47.089768 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 385/500] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:44:46 2022] GET /compute/v2.1/servers/f6051269-1d16-42db-8323-67a532dbcf40 => generated 1737 bytes in 196 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 19 00:44:47.178304 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-08cc8da4-6de8-4af2-8d99-e590efff647f tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:44:47.179003 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-08cc8da4-6de8-4af2-8d99-e590efff647f tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] [instance: d8ff0e88-597f-4db1-b5f4-852dd33c566d] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:44:47.181186 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG dbcounter [-] [125641] Writing DB stats nova_api:SELECT=94,nova_api:INSERT=6 {{(pid=125641) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 00:44:47.187655 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-08cc8da4-6de8-4af2-8d99-e590efff647f tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:47.188145 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-08cc8da4-6de8-4af2-8d99-e590efff647f tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:47.188581 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-08cc8da4-6de8-4af2-8d99-e590efff647f tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:47.189310 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-08cc8da4-6de8-4af2-8d99-e590efff647f tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:47.189765 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-08cc8da4-6de8-4af2-8d99-e590efff647f tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:47.190186 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-08cc8da4-6de8-4af2-8d99-e590efff647f tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:47.343292 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-08cc8da4-6de8-4af2-8d99-e590efff647f tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f8725c4d05b4dd79b05b43fba15980b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0b35c2038f6b4d0a9012b004f1416c00', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:44:47.352033 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-08cc8da4-6de8-4af2-8d99-e590efff647f tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] 10.222.0.26 "GET /compute/v2.1/servers/d8ff0e88-597f-4db1-b5f4-852dd33c566d" status: 200 len: 1367 microversion: 2.1 time: 0.177824 Oct 19 00:44:47.352623 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 386/501] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:44:47 2022] GET /compute/v2.1/servers/d8ff0e88-597f-4db1-b5f4-852dd33c566d => generated 1367 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 19 00:44:47.420517 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-aaef4c12-b046-42a0-aab0-f7e1ed627000 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:44:47.421147 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-aaef4c12-b046-42a0-aab0-f7e1ed627000 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] [instance: 0b7c19ee-6d89-4855-bf4b-d35b65bb170b] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:44:47.427592 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-aaef4c12-b046-42a0-aab0-f7e1ed627000 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:47.428089 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-aaef4c12-b046-42a0-aab0-f7e1ed627000 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:47.428527 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-aaef4c12-b046-42a0-aab0-f7e1ed627000 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:47.429243 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-aaef4c12-b046-42a0-aab0-f7e1ed627000 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:47.429708 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-aaef4c12-b046-42a0-aab0-f7e1ed627000 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:47.430167 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-aaef4c12-b046-42a0-aab0-f7e1ed627000 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:47.584994 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-aaef4c12-b046-42a0-aab0-f7e1ed627000 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f738e67d8c2340028da1254b60d7810e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8684504b0eff415b9063cebae21ca667', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:44:47.594551 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-aaef4c12-b046-42a0-aab0-f7e1ed627000 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] 10.222.0.26 "GET /compute/v2.1/servers/0b7c19ee-6d89-4855-bf4b-d35b65bb170b" status: 200 len: 1266 microversion: 2.1 time: 0.178814 Oct 19 00:44:47.595322 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 387/502] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:44:47 2022] GET /compute/v2.1/servers/0b7c19ee-6d89-4855-bf4b-d35b65bb170b => generated 1266 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 19 00:44:47.600021 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-46a590ba-7500-46ea-a6e1-a799359fa000 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:44:47.600636 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-46a590ba-7500-46ea-a6e1-a799359fa000 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] [instance: c7dd981d-4382-47dc-8d9c-effae6f630f3] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:44:47.607785 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-46a590ba-7500-46ea-a6e1-a799359fa000 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:47.608541 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-46a590ba-7500-46ea-a6e1-a799359fa000 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:47.609058 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-46a590ba-7500-46ea-a6e1-a799359fa000 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:47.610024 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-46a590ba-7500-46ea-a6e1-a799359fa000 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:47.610634 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-46a590ba-7500-46ea-a6e1-a799359fa000 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:47.611170 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-46a590ba-7500-46ea-a6e1-a799359fa000 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:47.680027 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-46a590ba-7500-46ea-a6e1-a799359fa000 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '39de6a6fff7c4cc08cc62aa86182d71d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1235d8b054b3405595fca761f08b73ae', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:44:47.694139 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-46a590ba-7500-46ea-a6e1-a799359fa000 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] 10.222.0.26 "GET /compute/v2.1/servers/c7dd981d-4382-47dc-8d9c-effae6f630f3" status: 200 len: 1222 microversion: 2.1 time: 0.097336 Oct 19 00:44:47.695765 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 388/503] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:44:47 2022] GET /compute/v2.1/servers/c7dd981d-4382-47dc-8d9c-effae6f630f3 => generated 1222 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:44:48.126027 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-0ebd727f-d711-4e40-b517-76d2c75af6f9 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:44:48.127223 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-0ebd727f-d711-4e40-b517-76d2c75af6f9 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] [instance: f6051269-1d16-42db-8323-67a532dbcf40] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:44:48.137040 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-0ebd727f-d711-4e40-b517-76d2c75af6f9 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:48.137732 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-0ebd727f-d711-4e40-b517-76d2c75af6f9 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:48.138577 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-0ebd727f-d711-4e40-b517-76d2c75af6f9 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:48.139740 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-0ebd727f-d711-4e40-b517-76d2c75af6f9 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:48.140464 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-0ebd727f-d711-4e40-b517-76d2c75af6f9 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:48.141177 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-0ebd727f-d711-4e40-b517-76d2c75af6f9 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:48.347975 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-0ebd727f-d711-4e40-b517-76d2c75af6f9 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd3f0d615187d4779bdd7da8c9f24c796', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0151d58abd047a692f5cc4e7d05bea5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:44:48.363338 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-0ebd727f-d711-4e40-b517-76d2c75af6f9 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] 10.222.0.26 "GET /compute/v2.1/servers/f6051269-1d16-42db-8323-67a532dbcf40" status: 200 len: 1737 microversion: 2.1 time: 0.240979 Oct 19 00:44:48.364485 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 389/504] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:44:48 2022] GET /compute/v2.1/servers/f6051269-1d16-42db-8323-67a532dbcf40 => generated 1737 bytes in 243 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 19 00:44:48.378443 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-28582c52-f772-40c4-8729-829b2c1c075c tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:44:48.379128 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-28582c52-f772-40c4-8729-829b2c1c075c tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] [instance: d8ff0e88-597f-4db1-b5f4-852dd33c566d] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:44:48.388731 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-28582c52-f772-40c4-8729-829b2c1c075c tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:48.389228 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-28582c52-f772-40c4-8729-829b2c1c075c tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:48.389675 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-28582c52-f772-40c4-8729-829b2c1c075c tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:48.390444 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-28582c52-f772-40c4-8729-829b2c1c075c tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:48.390949 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-28582c52-f772-40c4-8729-829b2c1c075c tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:48.391411 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-28582c52-f772-40c4-8729-829b2c1c075c tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:48.548118 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-28582c52-f772-40c4-8729-829b2c1c075c tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f8725c4d05b4dd79b05b43fba15980b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0b35c2038f6b4d0a9012b004f1416c00', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:44:48.561525 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-28582c52-f772-40c4-8729-829b2c1c075c tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] 10.222.0.26 "GET /compute/v2.1/servers/d8ff0e88-597f-4db1-b5f4-852dd33c566d" status: 200 len: 1367 microversion: 2.1 time: 0.187909 Oct 19 00:44:48.562370 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 390/505] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:44:48 2022] GET /compute/v2.1/servers/d8ff0e88-597f-4db1-b5f4-852dd33c566d => generated 1367 bytes in 190 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:44:48.618384 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-9a9ed958-4010-45bf-8674-bb6f4a93af46 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:44:48.619791 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-9a9ed958-4010-45bf-8674-bb6f4a93af46 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] [instance: 0b7c19ee-6d89-4855-bf4b-d35b65bb170b] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:44:48.626471 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-9a9ed958-4010-45bf-8674-bb6f4a93af46 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:48.627146 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-9a9ed958-4010-45bf-8674-bb6f4a93af46 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:48.627660 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-9a9ed958-4010-45bf-8674-bb6f4a93af46 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:48.628450 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-9a9ed958-4010-45bf-8674-bb6f4a93af46 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:48.628914 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-9a9ed958-4010-45bf-8674-bb6f4a93af46 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:48.629352 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-9a9ed958-4010-45bf-8674-bb6f4a93af46 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:48.798720 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-9a9ed958-4010-45bf-8674-bb6f4a93af46 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f738e67d8c2340028da1254b60d7810e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8684504b0eff415b9063cebae21ca667', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:44:48.807180 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-9a9ed958-4010-45bf-8674-bb6f4a93af46 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] 10.222.0.26 "GET /compute/v2.1/servers/0b7c19ee-6d89-4855-bf4b-d35b65bb170b" status: 200 len: 1266 microversion: 2.1 time: 0.193029 Oct 19 00:44:48.807768 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 391/506] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:44:48 2022] GET /compute/v2.1/servers/0b7c19ee-6d89-4855-bf4b-d35b65bb170b => generated 1266 bytes in 194 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 19 00:44:48.812638 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-efdc50ba-cca1-4f2d-9876-607f9d3767c0 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:44:48.813323 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-efdc50ba-cca1-4f2d-9876-607f9d3767c0 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] [instance: c7dd981d-4382-47dc-8d9c-effae6f630f3] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:44:48.820265 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-efdc50ba-cca1-4f2d-9876-607f9d3767c0 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:48.820756 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-efdc50ba-cca1-4f2d-9876-607f9d3767c0 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:48.821189 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-efdc50ba-cca1-4f2d-9876-607f9d3767c0 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:48.822350 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-efdc50ba-cca1-4f2d-9876-607f9d3767c0 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:48.822867 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-efdc50ba-cca1-4f2d-9876-607f9d3767c0 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:48.823293 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-efdc50ba-cca1-4f2d-9876-607f9d3767c0 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:48.893813 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-efdc50ba-cca1-4f2d-9876-607f9d3767c0 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '39de6a6fff7c4cc08cc62aa86182d71d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1235d8b054b3405595fca761f08b73ae', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:44:48.902461 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-efdc50ba-cca1-4f2d-9876-607f9d3767c0 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] 10.222.0.26 "GET /compute/v2.1/servers/c7dd981d-4382-47dc-8d9c-effae6f630f3" status: 200 len: 1214 microversion: 2.1 time: 0.093481 Oct 19 00:44:48.903099 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 392/507] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:44:48 2022] GET /compute/v2.1/servers/c7dd981d-4382-47dc-8d9c-effae6f630f3 => generated 1214 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:44:49.399329 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-7e8ddb52-41e1-4f34-8d4a-719634ee7a95 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:44:49.400580 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-7e8ddb52-41e1-4f34-8d4a-719634ee7a95 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] [instance: f6051269-1d16-42db-8323-67a532dbcf40] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:44:49.408054 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-7e8ddb52-41e1-4f34-8d4a-719634ee7a95 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:49.408544 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-7e8ddb52-41e1-4f34-8d4a-719634ee7a95 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:49.408977 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-7e8ddb52-41e1-4f34-8d4a-719634ee7a95 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:49.409688 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-7e8ddb52-41e1-4f34-8d4a-719634ee7a95 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:49.410140 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-7e8ddb52-41e1-4f34-8d4a-719634ee7a95 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:49.410595 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-7e8ddb52-41e1-4f34-8d4a-719634ee7a95 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:49.604156 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-7e8ddb52-41e1-4f34-8d4a-719634ee7a95 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd3f0d615187d4779bdd7da8c9f24c796', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0151d58abd047a692f5cc4e7d05bea5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:44:49.619371 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-7e8ddb52-41e1-4f34-8d4a-719634ee7a95 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] 10.222.0.26 "GET /compute/v2.1/servers/f6051269-1d16-42db-8323-67a532dbcf40" status: 200 len: 1737 microversion: 2.1 time: 0.223582 Oct 19 00:44:49.620283 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 393/508] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:44:49 2022] GET /compute/v2.1/servers/f6051269-1d16-42db-8323-67a532dbcf40 => generated 1737 bytes in 226 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 19 00:44:49.628664 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-eabdcaae-96ae-4f31-aced-29df9ceb385d tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:44:49.629662 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-eabdcaae-96ae-4f31-aced-29df9ceb385d tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] [instance: d8ff0e88-597f-4db1-b5f4-852dd33c566d] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:44:49.641456 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-eabdcaae-96ae-4f31-aced-29df9ceb385d tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:49.642324 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-eabdcaae-96ae-4f31-aced-29df9ceb385d tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:49.643137 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-eabdcaae-96ae-4f31-aced-29df9ceb385d tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:49.644759 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-eabdcaae-96ae-4f31-aced-29df9ceb385d tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:49.645406 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-eabdcaae-96ae-4f31-aced-29df9ceb385d tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:49.646168 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-eabdcaae-96ae-4f31-aced-29df9ceb385d tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:49.832956 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-eabdcaae-96ae-4f31-aced-29df9ceb385d tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f8725c4d05b4dd79b05b43fba15980b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0b35c2038f6b4d0a9012b004f1416c00', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:44:49.850779 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-eabdcaae-96ae-4f31-aced-29df9ceb385d tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] 10.222.0.26 "GET /compute/v2.1/servers/d8ff0e88-597f-4db1-b5f4-852dd33c566d" status: 200 len: 1367 microversion: 2.1 time: 0.228637 Oct 19 00:44:49.851775 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 394/509] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:44:49 2022] GET /compute/v2.1/servers/d8ff0e88-597f-4db1-b5f4-852dd33c566d => generated 1367 bytes in 231 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:44:49.860007 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-b9a4b657-ed0c-4478-b5a5-ef8be72600c6 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:44:49.861027 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-b9a4b657-ed0c-4478-b5a5-ef8be72600c6 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] [instance: 0b7c19ee-6d89-4855-bf4b-d35b65bb170b] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:44:49.874732 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-b9a4b657-ed0c-4478-b5a5-ef8be72600c6 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:49.875543 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-b9a4b657-ed0c-4478-b5a5-ef8be72600c6 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:49.876283 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-b9a4b657-ed0c-4478-b5a5-ef8be72600c6 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:49.877472 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-b9a4b657-ed0c-4478-b5a5-ef8be72600c6 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:49.878237 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-b9a4b657-ed0c-4478-b5a5-ef8be72600c6 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:49.879015 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-b9a4b657-ed0c-4478-b5a5-ef8be72600c6 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:49.903720 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG dbcounter [-] [125641] Writing DB stats nova_cell1:SELECT=100 {{(pid=125641) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 00:44:50.040301 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-b9a4b657-ed0c-4478-b5a5-ef8be72600c6 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f738e67d8c2340028da1254b60d7810e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8684504b0eff415b9063cebae21ca667', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:44:50.052442 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-b9a4b657-ed0c-4478-b5a5-ef8be72600c6 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] 10.222.0.26 "GET /compute/v2.1/servers/0b7c19ee-6d89-4855-bf4b-d35b65bb170b" status: 200 len: 1266 microversion: 2.1 time: 0.197805 Oct 19 00:44:50.053460 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 395/510] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:44:49 2022] GET /compute/v2.1/servers/0b7c19ee-6d89-4855-bf4b-d35b65bb170b => generated 1266 bytes in 201 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:44:50.062571 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-cac82a09-6887-4da9-964f-8ebdab5050e8 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:44:50.063615 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-cac82a09-6887-4da9-964f-8ebdab5050e8 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] [instance: c7dd981d-4382-47dc-8d9c-effae6f630f3] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:44:50.072780 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-cac82a09-6887-4da9-964f-8ebdab5050e8 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:50.073375 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-cac82a09-6887-4da9-964f-8ebdab5050e8 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:50.073826 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-cac82a09-6887-4da9-964f-8ebdab5050e8 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:50.074602 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-cac82a09-6887-4da9-964f-8ebdab5050e8 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:50.075093 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-cac82a09-6887-4da9-964f-8ebdab5050e8 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:50.075530 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-cac82a09-6887-4da9-964f-8ebdab5050e8 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:50.146266 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-cac82a09-6887-4da9-964f-8ebdab5050e8 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '39de6a6fff7c4cc08cc62aa86182d71d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1235d8b054b3405595fca761f08b73ae', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:44:50.161594 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-cac82a09-6887-4da9-964f-8ebdab5050e8 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] 10.222.0.26 "GET /compute/v2.1/servers/c7dd981d-4382-47dc-8d9c-effae6f630f3" status: 200 len: 1270 microversion: 2.1 time: 0.106169 Oct 19 00:44:50.162679 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 396/511] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:44:50 2022] GET /compute/v2.1/servers/c7dd981d-4382-47dc-8d9c-effae6f630f3 => generated 1270 bytes in 109 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:44:50.650973 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-d79395b1-2d9c-46dd-a847-88a8e83df649 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:44:50.652368 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-d79395b1-2d9c-46dd-a847-88a8e83df649 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] [instance: f6051269-1d16-42db-8323-67a532dbcf40] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:44:50.660440 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d79395b1-2d9c-46dd-a847-88a8e83df649 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:50.661170 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d79395b1-2d9c-46dd-a847-88a8e83df649 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:50.661811 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d79395b1-2d9c-46dd-a847-88a8e83df649 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:50.662780 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d79395b1-2d9c-46dd-a847-88a8e83df649 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:50.663422 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d79395b1-2d9c-46dd-a847-88a8e83df649 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:50.664281 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d79395b1-2d9c-46dd-a847-88a8e83df649 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:50.876342 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-d79395b1-2d9c-46dd-a847-88a8e83df649 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd3f0d615187d4779bdd7da8c9f24c796', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0151d58abd047a692f5cc4e7d05bea5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:44:50.885744 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-d79395b1-2d9c-46dd-a847-88a8e83df649 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] 10.222.0.26 "GET /compute/v2.1/servers/f6051269-1d16-42db-8323-67a532dbcf40" status: 200 len: 1717 microversion: 2.1 time: 0.240755 Oct 19 00:44:50.886755 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 397/512] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:44:50 2022] GET /compute/v2.1/servers/f6051269-1d16-42db-8323-67a532dbcf40 => generated 1717 bytes in 243 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 19 00:44:50.892766 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [req-f58c0336-75e3-449b-86bf-90a3fdabc761 req-715e54d9-d389-4ccc-be64-a246f596b105 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "d8ff0e88-597f-4db1-b5f4-852dd33c566d", "name": "network-vif-plugged", "status": "completed", "tag": "1598d129-803e-437a-be2e-c9127e38bd88"}]} {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 19 00:44:50.904740 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-f58c0336-75e3-449b-86bf-90a3fdabc761 req-715e54d9-d389-4ccc-be64-a246f596b105 service nova] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:50.905236 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-f58c0336-75e3-449b-86bf-90a3fdabc761 req-715e54d9-d389-4ccc-be64-a246f596b105 service nova] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:50.905678 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-f58c0336-75e3-449b-86bf-90a3fdabc761 req-715e54d9-d389-4ccc-be64-a246f596b105 service nova] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:50.919474 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.compute.server_external_events [req-f58c0336-75e3-449b-86bf-90a3fdabc761 req-715e54d9-d389-4ccc-be64-a246f596b105 service nova] Creating event network-vif-plugged:1598d129-803e-437a-be2e-c9127e38bd88 for instance d8ff0e88-597f-4db1-b5f4-852dd33c566d on hv-cinder-maste Oct 19 00:44:50.924845 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [req-f58c0336-75e3-449b-86bf-90a3fdabc761 req-715e54d9-d389-4ccc-be64-a246f596b105 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.036050 Oct 19 00:44:50.925459 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 398/513] 10.222.0.26 () {64 vars in 1317 bytes} [Wed Oct 19 00:44:50 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 19 00:44:50.931040 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-8bc64d8c-fc7b-4341-bcae-6118fad0e02b tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:44:50.931674 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-8bc64d8c-fc7b-4341-bcae-6118fad0e02b tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] [instance: d8ff0e88-597f-4db1-b5f4-852dd33c566d] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:44:50.938733 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-8bc64d8c-fc7b-4341-bcae-6118fad0e02b tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:50.939228 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-8bc64d8c-fc7b-4341-bcae-6118fad0e02b tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:50.939662 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-8bc64d8c-fc7b-4341-bcae-6118fad0e02b tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:50.940391 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-8bc64d8c-fc7b-4341-bcae-6118fad0e02b tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:50.940844 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-8bc64d8c-fc7b-4341-bcae-6118fad0e02b tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:50.941268 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-8bc64d8c-fc7b-4341-bcae-6118fad0e02b tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:51.132503 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-8bc64d8c-fc7b-4341-bcae-6118fad0e02b tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f8725c4d05b4dd79b05b43fba15980b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0b35c2038f6b4d0a9012b004f1416c00', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:44:51.149900 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-8bc64d8c-fc7b-4341-bcae-6118fad0e02b tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] 10.222.0.26 "GET /compute/v2.1/servers/d8ff0e88-597f-4db1-b5f4-852dd33c566d" status: 200 len: 1367 microversion: 2.1 time: 0.222546 Oct 19 00:44:51.151653 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 399/514] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:44:50 2022] GET /compute/v2.1/servers/d8ff0e88-597f-4db1-b5f4-852dd33c566d => generated 1367 bytes in 225 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:44:51.162184 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-504b3274-b509-4373-8285-7bd50e5908fd tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:44:51.163293 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-504b3274-b509-4373-8285-7bd50e5908fd tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] [instance: f6051269-1d16-42db-8323-67a532dbcf40] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:44:51.175981 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-504b3274-b509-4373-8285-7bd50e5908fd tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:51.176955 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-504b3274-b509-4373-8285-7bd50e5908fd tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:51.177861 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-504b3274-b509-4373-8285-7bd50e5908fd tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:51.179489 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-504b3274-b509-4373-8285-7bd50e5908fd tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:51.180605 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-504b3274-b509-4373-8285-7bd50e5908fd tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:51.182081 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-504b3274-b509-4373-8285-7bd50e5908fd tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:51.328684 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.wsgi [None req-504b3274-b509-4373-8285-7bd50e5908fd tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] HTTP exception thrown: Cannot 'detach_volume' instance f6051269-1d16-42db-8323-67a532dbcf40 while it is in vm_state rescued Oct 19 00:44:51.330723 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-504b3274-b509-4373-8285-7bd50e5908fd tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Request method failure captured: Oct 19 00:44:51.330723 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: request: DELETE /compute/v2.1/servers/f6051269-1d16-42db-8323-67a532dbcf40/os-volume_attachments/b9abcc63-2fc1-4c36-a503-af404a15c9f5 HTTP/1.1 Oct 19 00:44:51.330723 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Accept: application/json Oct 19 00:44:51.330723 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Accept-Encoding: identity Oct 19 00:44:51.330723 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Connection: close Oct 19 00:44:51.330723 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Content-Length: 0 Oct 19 00:44:51.330723 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Content-Type: application/json Oct 19 00:44:51.330723 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Host: 10.222.0.26 Oct 19 00:44:51.330723 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Openstack-System-Scope: None Oct 19 00:44:51.330723 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: User-Agent: python-urllib3/1.26.12 Oct 19 00:44:51.330723 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Auth-Token: gAAAAABjT0eG-h4KFbZ30OiVtvJ2IwzaIfjKNSU1Qsat-O9_Zg1Ouctw-AlAuBWD6kHGuP3EnbU_TJPE1ljWwW1kJo03Mn42ZE5duN2HDm9MXxFr1Oc_Mse4KR4ABtf0tvzP53Pz6nUn07toZIxQKWNLC4yurNeR_6sS4VV90-4qptRCmdstdYU Oct 19 00:44:51.330723 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Domain-Id: None Oct 19 00:44:51.330723 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Domain-Name: None Oct 19 00:44:51.330723 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Identity-Status: Confirmed Oct 19 00:44:51.330723 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Is-Admin-Project: True Oct 19 00:44:51.330723 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Project-Domain-Id: default Oct 19 00:44:51.330723 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Project-Domain-Name: Default Oct 19 00:44:51.330723 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Project-Id: c0151d58abd047a692f5cc4e7d05bea5 Oct 19 00:44:51.330723 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-190719564 Oct 19 00:44:51.330723 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Role: member,reader Oct 19 00:44:51.330723 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Roles: member,reader Oct 19 00:44:51.333523 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/c0151d58abd047a692f5cc4e7d05bea5"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/c0151d58abd047a692f5cc4e7d05bea5"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/c0151d58abd047a692f5cc4e7d05bea5"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_c0151d58abd047a692f5cc4e7d05bea5"}]}] Oct 19 00:44:51.333523 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Tenant: tempest-ServerRescueNegativeTestJSON-190719564 Oct 19 00:44:51.333523 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Tenant-Id: c0151d58abd047a692f5cc4e7d05bea5 Oct 19 00:44:51.333523 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-190719564 Oct 19 00:44:51.333523 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User: tempest-ServerRescueNegativeTestJSON-190719564-project Oct 19 00:44:51.333523 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User-Domain-Id: default Oct 19 00:44:51.333523 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User-Domain-Name: Default Oct 19 00:44:51.333523 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User-Id: d3f0d615187d4779bdd7da8c9f24c796 Oct 19 00:44:51.333523 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User-Name: tempest-ServerRescueNegativeTestJSON-190719564-project Oct 19 00:44:51.333523 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: method: > Oct 19 00:44:51.333523 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: exception: Cannot 'detach_volume' instance f6051269-1d16-42db-8323-67a532dbcf40 while it is in vm_state rescued Oct 19 00:44:51.334543 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: action_args: {'server_id': 'f6051269-1d16-42db-8323-67a532dbcf40', 'id': 'b9abcc63-2fc1-4c36-a503-af404a15c9f5'} Oct 19 00:44:51.334543 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 19 00:44:51.334543 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 19 00:44:51.334543 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete Oct 19 00:44:51.334543 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) Oct 19 00:44:51.334543 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 241, in inner Oct 19 00:44:51.334543 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Oct 19 00:44:51.334543 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 148, in inner Oct 19 00:44:51.334543 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi raise exception.InstanceInvalidState( Oct 19 00:44:51.334543 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi nova.exception.InstanceInvalidState: Instance f6051269-1d16-42db-8323-67a532dbcf40 in vm_state rescued. Cannot detach_volume while the instance is in this state. Oct 19 00:44:51.334543 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:44:51.334543 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 19 00:44:51.334543 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:44:51.334543 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 19 00:44:51.334543 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 19 00:44:51.334543 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 19 00:44:51.334543 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 19 00:44:51.334543 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 19 00:44:51.334543 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 19 00:44:51.334543 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 19 00:44:51.336811 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 573, in delete Oct 19 00:44:51.336811 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi common.raise_http_conflict_for_instance_invalid_state(state_error, Oct 19 00:44:51.336811 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 347, in raise_http_conflict_for_instance_invalid_state Oct 19 00:44:51.336811 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPConflict(explanation=msg) Oct 19 00:44:51.336811 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi webob.exc.HTTPConflict: Cannot 'detach_volume' instance f6051269-1d16-42db-8323-67a532dbcf40 while it is in vm_state rescued Oct 19 00:44:51.336811 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:44:51.336811 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 19 00:44:51.336811 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:44:51.336811 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 19 00:44:51.336811 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 19 00:44:51.336811 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 19 00:44:51.336811 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 19 00:44:51.336811 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 19 00:44:51.336811 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Cannot 'detach_volume' instance f6051269-1d16-42db-8323-67a532dbcf40 while it is in vm_state rescued Oct 19 00:44:51.336811 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:44:51.336811 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-504b3274-b509-4373-8285-7bd50e5908fd tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Returning 409 to user: Cannot 'detach_volume' instance f6051269-1d16-42db-8323-67a532dbcf40 while it is in vm_state rescued {{(pid=125641) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 19 00:44:51.339416 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-504b3274-b509-4373-8285-7bd50e5908fd tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] 10.222.0.26 "DELETE /compute/v2.1/servers/f6051269-1d16-42db-8323-67a532dbcf40/os-volume_attachments/b9abcc63-2fc1-4c36-a503-af404a15c9f5" status: 409 len: 152 microversion: 2.1 time: 0.178937 Oct 19 00:44:51.339416 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 400/515] 10.222.0.26 () {58 vars in 1416 bytes} [Wed Oct 19 00:44:51 2022] DELETE /compute/v2.1/servers/f6051269-1d16-42db-8323-67a532dbcf40/os-volume_attachments/b9abcc63-2fc1-4c36-a503-af404a15c9f5 => generated 152 bytes in 181 msecs (HTTP/1.1 409) 9 headers in 378 bytes (1 switches on core 0) Oct 19 00:44:51.350231 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-5898b5ed-1617-45c6-90e1-b746a8cfc2f0 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:44:51.351037 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-5898b5ed-1617-45c6-90e1-b746a8cfc2f0 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] [instance: 0b7c19ee-6d89-4855-bf4b-d35b65bb170b] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:44:51.359450 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-5898b5ed-1617-45c6-90e1-b746a8cfc2f0 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:51.360146 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-5898b5ed-1617-45c6-90e1-b746a8cfc2f0 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:51.360855 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-5898b5ed-1617-45c6-90e1-b746a8cfc2f0 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:51.361983 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-5898b5ed-1617-45c6-90e1-b746a8cfc2f0 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:51.362657 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-5898b5ed-1617-45c6-90e1-b746a8cfc2f0 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:51.363256 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-5898b5ed-1617-45c6-90e1-b746a8cfc2f0 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:51.520722 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-5898b5ed-1617-45c6-90e1-b746a8cfc2f0 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f738e67d8c2340028da1254b60d7810e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8684504b0eff415b9063cebae21ca667', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:44:51.529759 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-5898b5ed-1617-45c6-90e1-b746a8cfc2f0 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] 10.222.0.26 "GET /compute/v2.1/servers/0b7c19ee-6d89-4855-bf4b-d35b65bb170b" status: 200 len: 1266 microversion: 2.1 time: 0.185945 Oct 19 00:44:51.530352 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 401/516] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:44:51 2022] GET /compute/v2.1/servers/0b7c19ee-6d89-4855-bf4b-d35b65bb170b => generated 1266 bytes in 196 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:44:51.535125 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-dc850883-a262-43d5-84f0-ad2e8f5239bd tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:44:51.535732 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-dc850883-a262-43d5-84f0-ad2e8f5239bd tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] [instance: c7dd981d-4382-47dc-8d9c-effae6f630f3] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:44:51.542893 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-dc850883-a262-43d5-84f0-ad2e8f5239bd tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:51.543682 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-dc850883-a262-43d5-84f0-ad2e8f5239bd tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:51.544335 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-dc850883-a262-43d5-84f0-ad2e8f5239bd tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:51.545233 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-dc850883-a262-43d5-84f0-ad2e8f5239bd tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:51.545898 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-dc850883-a262-43d5-84f0-ad2e8f5239bd tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:51.546683 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-dc850883-a262-43d5-84f0-ad2e8f5239bd tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:51.617961 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-dc850883-a262-43d5-84f0-ad2e8f5239bd tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '39de6a6fff7c4cc08cc62aa86182d71d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1235d8b054b3405595fca761f08b73ae', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:44:51.626629 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-dc850883-a262-43d5-84f0-ad2e8f5239bd tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] 10.222.0.26 "GET /compute/v2.1/servers/c7dd981d-4382-47dc-8d9c-effae6f630f3" status: 200 len: 1270 microversion: 2.1 time: 0.095068 Oct 19 00:44:51.627133 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 402/517] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:44:51 2022] GET /compute/v2.1/servers/c7dd981d-4382-47dc-8d9c-effae6f630f3 => generated 1270 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:44:51.632045 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-da7626a8-ef6f-4567-871e-2b6aeeea84dd tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Action: 'action', calling method: >, body: {"unrescue": {}} {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 19 00:44:51.632584 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-da7626a8-ef6f-4567-871e-2b6aeeea84dd tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] [instance: f6051269-1d16-42db-8323-67a532dbcf40] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:44:51.639498 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-da7626a8-ef6f-4567-871e-2b6aeeea84dd tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:51.639992 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-da7626a8-ef6f-4567-871e-2b6aeeea84dd tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:51.640434 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-da7626a8-ef6f-4567-871e-2b6aeeea84dd tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:51.641146 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-da7626a8-ef6f-4567-871e-2b6aeeea84dd tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:51.641608 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-da7626a8-ef6f-4567-871e-2b6aeeea84dd tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:51.642038 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-da7626a8-ef6f-4567-871e-2b6aeeea84dd tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:51.717119 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-da7626a8-ef6f-4567-871e-2b6aeeea84dd tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] 10.222.0.26 "POST /compute/v2.1/servers/f6051269-1d16-42db-8323-67a532dbcf40/action" status: 202 len: 0 microversion: 2.1 time: 0.088641 Oct 19 00:44:51.717872 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 403/518] 10.222.0.26 () {60 vars in 1278 bytes} [Wed Oct 19 00:44:51 2022] POST /compute/v2.1/servers/f6051269-1d16-42db-8323-67a532dbcf40/action => generated 0 bytes in 90 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 19 00:44:51.726855 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-27ab0a63-3b85-433a-95c9-399c67b97fa6 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:44:51.727576 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-27ab0a63-3b85-433a-95c9-399c67b97fa6 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] [instance: f6051269-1d16-42db-8323-67a532dbcf40] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:44:51.734208 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-27ab0a63-3b85-433a-95c9-399c67b97fa6 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:51.734739 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-27ab0a63-3b85-433a-95c9-399c67b97fa6 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:51.735183 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-27ab0a63-3b85-433a-95c9-399c67b97fa6 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:51.735908 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-27ab0a63-3b85-433a-95c9-399c67b97fa6 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:51.736368 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-27ab0a63-3b85-433a-95c9-399c67b97fa6 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:51.736797 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-27ab0a63-3b85-433a-95c9-399c67b97fa6 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:51.885738 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-27ab0a63-3b85-433a-95c9-399c67b97fa6 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd3f0d615187d4779bdd7da8c9f24c796', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0151d58abd047a692f5cc4e7d05bea5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:44:51.900729 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-27ab0a63-3b85-433a-95c9-399c67b97fa6 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] 10.222.0.26 "GET /compute/v2.1/servers/f6051269-1d16-42db-8323-67a532dbcf40" status: 200 len: 1725 microversion: 2.1 time: 0.177127 Oct 19 00:44:51.901776 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 404/519] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:44:51 2022] GET /compute/v2.1/servers/f6051269-1d16-42db-8323-67a532dbcf40 => generated 1725 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 19 00:44:52.177000 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-87128a6e-2a19-415e-a03d-ee1510008985 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:44:52.177523 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-87128a6e-2a19-415e-a03d-ee1510008985 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] [instance: d8ff0e88-597f-4db1-b5f4-852dd33c566d] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:44:52.184298 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-87128a6e-2a19-415e-a03d-ee1510008985 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:52.184788 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-87128a6e-2a19-415e-a03d-ee1510008985 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:52.185217 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-87128a6e-2a19-415e-a03d-ee1510008985 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:52.185937 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-87128a6e-2a19-415e-a03d-ee1510008985 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:52.186387 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-87128a6e-2a19-415e-a03d-ee1510008985 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:52.186842 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-87128a6e-2a19-415e-a03d-ee1510008985 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:52.340812 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-87128a6e-2a19-415e-a03d-ee1510008985 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f8725c4d05b4dd79b05b43fba15980b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0b35c2038f6b4d0a9012b004f1416c00', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:44:52.349532 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-87128a6e-2a19-415e-a03d-ee1510008985 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] 10.222.0.26 "GET /compute/v2.1/servers/d8ff0e88-597f-4db1-b5f4-852dd33c566d" status: 200 len: 1367 microversion: 2.1 time: 0.176934 Oct 19 00:44:52.350187 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 405/520] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:44:52 2022] GET /compute/v2.1/servers/d8ff0e88-597f-4db1-b5f4-852dd33c566d => generated 1367 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 19 00:44:52.562248 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-6e67f5db-2a16-461f-aeae-fffceff5c404 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:44:52.563243 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-6e67f5db-2a16-461f-aeae-fffceff5c404 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] [instance: 0b7c19ee-6d89-4855-bf4b-d35b65bb170b] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:44:52.569974 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-6e67f5db-2a16-461f-aeae-fffceff5c404 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:52.570545 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-6e67f5db-2a16-461f-aeae-fffceff5c404 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:52.571055 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-6e67f5db-2a16-461f-aeae-fffceff5c404 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:52.572406 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-6e67f5db-2a16-461f-aeae-fffceff5c404 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:52.573432 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-6e67f5db-2a16-461f-aeae-fffceff5c404 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:52.573432 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-6e67f5db-2a16-461f-aeae-fffceff5c404 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:52.863582 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-6e67f5db-2a16-461f-aeae-fffceff5c404 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f738e67d8c2340028da1254b60d7810e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8684504b0eff415b9063cebae21ca667', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:44:52.884887 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-6e67f5db-2a16-461f-aeae-fffceff5c404 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] 10.222.0.26 "GET /compute/v2.1/servers/0b7c19ee-6d89-4855-bf4b-d35b65bb170b" status: 200 len: 1266 microversion: 2.1 time: 0.329769 Oct 19 00:44:52.888501 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 406/521] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:44:52 2022] GET /compute/v2.1/servers/0b7c19ee-6d89-4855-bf4b-d35b65bb170b => generated 1266 bytes in 335 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:44:52.895221 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-44c8fbba-5ef7-4c5e-bc10-f687a81521e5 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:44:52.896686 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-44c8fbba-5ef7-4c5e-bc10-f687a81521e5 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] [instance: c7dd981d-4382-47dc-8d9c-effae6f630f3] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:44:52.925196 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-44c8fbba-5ef7-4c5e-bc10-f687a81521e5 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:52.926016 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-44c8fbba-5ef7-4c5e-bc10-f687a81521e5 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:52.926822 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-44c8fbba-5ef7-4c5e-bc10-f687a81521e5 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:52.928997 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-44c8fbba-5ef7-4c5e-bc10-f687a81521e5 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:52.932866 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-44c8fbba-5ef7-4c5e-bc10-f687a81521e5 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:52.935504 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-44c8fbba-5ef7-4c5e-bc10-f687a81521e5 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:53.072715 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-44c8fbba-5ef7-4c5e-bc10-f687a81521e5 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '39de6a6fff7c4cc08cc62aa86182d71d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1235d8b054b3405595fca761f08b73ae', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:44:53.082203 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-44c8fbba-5ef7-4c5e-bc10-f687a81521e5 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] 10.222.0.26 "GET /compute/v2.1/servers/c7dd981d-4382-47dc-8d9c-effae6f630f3" status: 200 len: 1276 microversion: 2.1 time: 0.192140 Oct 19 00:44:53.082854 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 407/522] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:44:52 2022] GET /compute/v2.1/servers/c7dd981d-4382-47dc-8d9c-effae6f630f3 => generated 1276 bytes in 194 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:44:53.088685 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-8c9aae25-b51f-4011-b490-f4089ffb6031 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:44:53.089310 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-8c9aae25-b51f-4011-b490-f4089ffb6031 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] [instance: f6051269-1d16-42db-8323-67a532dbcf40] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:44:53.096450 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-8c9aae25-b51f-4011-b490-f4089ffb6031 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:53.096941 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-8c9aae25-b51f-4011-b490-f4089ffb6031 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:53.097412 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-8c9aae25-b51f-4011-b490-f4089ffb6031 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:53.098093 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-8c9aae25-b51f-4011-b490-f4089ffb6031 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:53.098601 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-8c9aae25-b51f-4011-b490-f4089ffb6031 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:53.099034 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-8c9aae25-b51f-4011-b490-f4089ffb6031 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:53.252793 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-8c9aae25-b51f-4011-b490-f4089ffb6031 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd3f0d615187d4779bdd7da8c9f24c796', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0151d58abd047a692f5cc4e7d05bea5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:44:53.262623 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-8c9aae25-b51f-4011-b490-f4089ffb6031 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] 10.222.0.26 "GET /compute/v2.1/servers/f6051269-1d16-42db-8323-67a532dbcf40" status: 200 len: 1725 microversion: 2.1 time: 0.178527 Oct 19 00:44:53.263467 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 408/523] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:44:53 2022] GET /compute/v2.1/servers/f6051269-1d16-42db-8323-67a532dbcf40 => generated 1725 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:44:53.374556 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-b0601479-7435-462b-a251-40dfaf57bd66 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:44:53.375117 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-b0601479-7435-462b-a251-40dfaf57bd66 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] [instance: d8ff0e88-597f-4db1-b5f4-852dd33c566d] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:44:53.381617 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-b0601479-7435-462b-a251-40dfaf57bd66 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:53.382118 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-b0601479-7435-462b-a251-40dfaf57bd66 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:53.382612 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-b0601479-7435-462b-a251-40dfaf57bd66 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:53.383379 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-b0601479-7435-462b-a251-40dfaf57bd66 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:53.383854 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-b0601479-7435-462b-a251-40dfaf57bd66 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:53.384285 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-b0601479-7435-462b-a251-40dfaf57bd66 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:53.564452 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-b0601479-7435-462b-a251-40dfaf57bd66 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f8725c4d05b4dd79b05b43fba15980b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0b35c2038f6b4d0a9012b004f1416c00', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:44:53.572801 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-b0601479-7435-462b-a251-40dfaf57bd66 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] 10.222.0.26 "GET /compute/v2.1/servers/d8ff0e88-597f-4db1-b5f4-852dd33c566d" status: 200 len: 1556 microversion: 2.1 time: 0.202579 Oct 19 00:44:53.574592 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 409/524] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:44:53 2022] GET /compute/v2.1/servers/d8ff0e88-597f-4db1-b5f4-852dd33c566d => generated 1556 bytes in 205 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 19 00:44:53.597706 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-0cc494a2-3dbb-479f-a25b-22544ef91264 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:44:53.598214 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-0cc494a2-3dbb-479f-a25b-22544ef91264 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] [instance: d8ff0e88-597f-4db1-b5f4-852dd33c566d] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:44:53.606860 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-0cc494a2-3dbb-479f-a25b-22544ef91264 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:53.607362 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-0cc494a2-3dbb-479f-a25b-22544ef91264 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:53.607807 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-0cc494a2-3dbb-479f-a25b-22544ef91264 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:53.608554 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-0cc494a2-3dbb-479f-a25b-22544ef91264 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:53.609021 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-0cc494a2-3dbb-479f-a25b-22544ef91264 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:53.609454 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-0cc494a2-3dbb-479f-a25b-22544ef91264 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:53.700616 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-0cc494a2-3dbb-479f-a25b-22544ef91264 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] 10.222.0.26 "GET /compute/v2.1/servers/d8ff0e88-597f-4db1-b5f4-852dd33c566d/os-interface" status: 200 len: 285 microversion: 2.1 time: 0.107192 Oct 19 00:44:53.706733 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 410/525] 10.222.0.26 () {58 vars in 1275 bytes} [Wed Oct 19 00:44:53 2022] GET /compute/v2.1/servers/d8ff0e88-597f-4db1-b5f4-852dd33c566d/os-interface => generated 285 bytes in 115 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 19 00:44:53.932235 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-618ecf5e-eb07-4f3f-9358-acf62b7902d3 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:44:53.935572 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-618ecf5e-eb07-4f3f-9358-acf62b7902d3 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] [instance: 0b7c19ee-6d89-4855-bf4b-d35b65bb170b] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:44:53.948365 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-618ecf5e-eb07-4f3f-9358-acf62b7902d3 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:53.949135 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-618ecf5e-eb07-4f3f-9358-acf62b7902d3 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:53.950179 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-618ecf5e-eb07-4f3f-9358-acf62b7902d3 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:53.952022 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-618ecf5e-eb07-4f3f-9358-acf62b7902d3 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:53.953387 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-618ecf5e-eb07-4f3f-9358-acf62b7902d3 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:53.954426 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-618ecf5e-eb07-4f3f-9358-acf62b7902d3 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:54.272691 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-618ecf5e-eb07-4f3f-9358-acf62b7902d3 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f738e67d8c2340028da1254b60d7810e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8684504b0eff415b9063cebae21ca667', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:44:54.293094 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-618ecf5e-eb07-4f3f-9358-acf62b7902d3 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] 10.222.0.26 "GET /compute/v2.1/servers/0b7c19ee-6d89-4855-bf4b-d35b65bb170b" status: 200 len: 1266 microversion: 2.1 time: 0.369530 Oct 19 00:44:54.294395 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 411/526] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:44:53 2022] GET /compute/v2.1/servers/0b7c19ee-6d89-4855-bf4b-d35b65bb170b => generated 1266 bytes in 372 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 19 00:44:54.317594 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-bcf59290-60a1-43bd-8ed9-22e1f7fe68fb tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:44:54.318253 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-bcf59290-60a1-43bd-8ed9-22e1f7fe68fb tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] [instance: c7dd981d-4382-47dc-8d9c-effae6f630f3] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:44:54.326398 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-bcf59290-60a1-43bd-8ed9-22e1f7fe68fb tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:54.326970 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-bcf59290-60a1-43bd-8ed9-22e1f7fe68fb tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:54.327452 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-bcf59290-60a1-43bd-8ed9-22e1f7fe68fb tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:54.328240 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-bcf59290-60a1-43bd-8ed9-22e1f7fe68fb tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:54.328725 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-bcf59290-60a1-43bd-8ed9-22e1f7fe68fb tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:54.330126 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-bcf59290-60a1-43bd-8ed9-22e1f7fe68fb tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:54.335609 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG dbcounter [-] [125641] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=125641) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 00:44:54.463864 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-bcf59290-60a1-43bd-8ed9-22e1f7fe68fb tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '39de6a6fff7c4cc08cc62aa86182d71d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1235d8b054b3405595fca761f08b73ae', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:44:54.473507 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-bcf59290-60a1-43bd-8ed9-22e1f7fe68fb tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] 10.222.0.26 "GET /compute/v2.1/servers/c7dd981d-4382-47dc-8d9c-effae6f630f3" status: 200 len: 1276 microversion: 2.1 time: 0.161118 Oct 19 00:44:54.473507 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 412/527] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:44:54 2022] GET /compute/v2.1/servers/c7dd981d-4382-47dc-8d9c-effae6f630f3 => generated 1276 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:44:54.494706 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-e7cb151f-27b3-4a1e-95e6-9cb32460bbfd tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:44:54.495755 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-e7cb151f-27b3-4a1e-95e6-9cb32460bbfd tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] [instance: f6051269-1d16-42db-8323-67a532dbcf40] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:44:54.502454 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-e7cb151f-27b3-4a1e-95e6-9cb32460bbfd tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:54.503204 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-e7cb151f-27b3-4a1e-95e6-9cb32460bbfd tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:54.503744 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-e7cb151f-27b3-4a1e-95e6-9cb32460bbfd tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:54.505098 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-e7cb151f-27b3-4a1e-95e6-9cb32460bbfd tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:54.505472 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-e7cb151f-27b3-4a1e-95e6-9cb32460bbfd tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:54.506001 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-e7cb151f-27b3-4a1e-95e6-9cb32460bbfd tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:54.837060 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-e7cb151f-27b3-4a1e-95e6-9cb32460bbfd tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd3f0d615187d4779bdd7da8c9f24c796', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0151d58abd047a692f5cc4e7d05bea5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:44:54.863528 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-e7cb151f-27b3-4a1e-95e6-9cb32460bbfd tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] 10.222.0.26 "GET /compute/v2.1/servers/f6051269-1d16-42db-8323-67a532dbcf40" status: 200 len: 1725 microversion: 2.1 time: 0.375159 Oct 19 00:44:54.865472 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 413/528] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:44:54 2022] GET /compute/v2.1/servers/f6051269-1d16-42db-8323-67a532dbcf40 => generated 1725 bytes in 391 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:44:54.883568 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [req-6b6fbe8d-b4a2-486c-a9c8-a8499e54bb13 req-6c84f18f-ea92-4b0c-94c1-b2eacb8153e6 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "0b7c19ee-6d89-4855-bf4b-d35b65bb170b", "name": "network-vif-plugged", "status": "completed", "tag": "3f92cbb3-bb62-457d-b0d7-92947884c5fb"}]} {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 19 00:44:54.911955 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-6b6fbe8d-b4a2-486c-a9c8-a8499e54bb13 req-6c84f18f-ea92-4b0c-94c1-b2eacb8153e6 service nova] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:54.914011 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-6b6fbe8d-b4a2-486c-a9c8-a8499e54bb13 req-6c84f18f-ea92-4b0c-94c1-b2eacb8153e6 service nova] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:54.918845 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-6b6fbe8d-b4a2-486c-a9c8-a8499e54bb13 req-6c84f18f-ea92-4b0c-94c1-b2eacb8153e6 service nova] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:54.942162 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.compute.server_external_events [req-6b6fbe8d-b4a2-486c-a9c8-a8499e54bb13 req-6c84f18f-ea92-4b0c-94c1-b2eacb8153e6 service nova] Creating event network-vif-plugged:3f92cbb3-bb62-457d-b0d7-92947884c5fb for instance 0b7c19ee-6d89-4855-bf4b-d35b65bb170b on hv-cinder-maste Oct 19 00:44:54.951280 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [req-6b6fbe8d-b4a2-486c-a9c8-a8499e54bb13 req-6c84f18f-ea92-4b0c-94c1-b2eacb8153e6 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.083181 Oct 19 00:44:54.952266 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 414/529] 10.222.0.26 () {64 vars in 1317 bytes} [Wed Oct 19 00:44:54 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 19 00:44:55.277403 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [req-303261e5-9928-44ce-829c-7d97b3a3a7df req-25c655e7-cd69-4fa2-aa52-365dc550e2a7 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "d8ff0e88-597f-4db1-b5f4-852dd33c566d", "tag": "1598d129-803e-437a-be2e-c9127e38bd88"}]} {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 19 00:44:55.289151 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-303261e5-9928-44ce-829c-7d97b3a3a7df req-25c655e7-cd69-4fa2-aa52-365dc550e2a7 service nova] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:55.289943 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-303261e5-9928-44ce-829c-7d97b3a3a7df req-25c655e7-cd69-4fa2-aa52-365dc550e2a7 service nova] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:55.291204 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-303261e5-9928-44ce-829c-7d97b3a3a7df req-25c655e7-cd69-4fa2-aa52-365dc550e2a7 service nova] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:55.321153 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.compute.server_external_events [req-303261e5-9928-44ce-829c-7d97b3a3a7df req-25c655e7-cd69-4fa2-aa52-365dc550e2a7 service nova] Creating event network-changed:1598d129-803e-437a-be2e-c9127e38bd88 for instance d8ff0e88-597f-4db1-b5f4-852dd33c566d on hv-cinder-maste Oct 19 00:44:55.328394 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [req-303261e5-9928-44ce-829c-7d97b3a3a7df req-25c655e7-cd69-4fa2-aa52-365dc550e2a7 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.056645 Oct 19 00:44:55.329444 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 415/530] 10.222.0.26 () {64 vars in 1317 bytes} [Wed Oct 19 00:44:55 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 59 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 19 00:44:55.335249 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-ca163897-4ca7-4629-b3e1-378de2c2e69d tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:44:55.336225 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-ca163897-4ca7-4629-b3e1-378de2c2e69d tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] [instance: 0b7c19ee-6d89-4855-bf4b-d35b65bb170b] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:44:55.345157 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ca163897-4ca7-4629-b3e1-378de2c2e69d tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:55.345672 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ca163897-4ca7-4629-b3e1-378de2c2e69d tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:55.346123 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ca163897-4ca7-4629-b3e1-378de2c2e69d tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:55.346993 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ca163897-4ca7-4629-b3e1-378de2c2e69d tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:55.347467 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ca163897-4ca7-4629-b3e1-378de2c2e69d tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:55.347901 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ca163897-4ca7-4629-b3e1-378de2c2e69d tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:55.526364 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG dbcounter [-] [125641] Writing DB stats nova_cell0:SELECT=2 {{(pid=125641) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 00:44:55.553180 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG dbcounter [-] [125641] Writing DB stats nova_cell0:SELECT=6 {{(pid=125641) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 00:44:55.568540 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-ca163897-4ca7-4629-b3e1-378de2c2e69d tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f738e67d8c2340028da1254b60d7810e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8684504b0eff415b9063cebae21ca667', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:44:55.583174 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-ca163897-4ca7-4629-b3e1-378de2c2e69d tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] 10.222.0.26 "GET /compute/v2.1/servers/0b7c19ee-6d89-4855-bf4b-d35b65bb170b" status: 200 len: 1266 microversion: 2.1 time: 0.252016 Oct 19 00:44:55.584859 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 416/531] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:44:55 2022] GET /compute/v2.1/servers/0b7c19ee-6d89-4855-bf4b-d35b65bb170b => generated 1266 bytes in 255 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:44:55.596005 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-8c4c62a2-3647-420e-9ef7-12a41c475b71 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:44:55.597173 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-8c4c62a2-3647-420e-9ef7-12a41c475b71 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] [instance: c7dd981d-4382-47dc-8d9c-effae6f630f3] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:44:55.609082 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-8c4c62a2-3647-420e-9ef7-12a41c475b71 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:55.609909 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-8c4c62a2-3647-420e-9ef7-12a41c475b71 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:55.610766 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-8c4c62a2-3647-420e-9ef7-12a41c475b71 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:55.611990 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-8c4c62a2-3647-420e-9ef7-12a41c475b71 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:55.612832 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-8c4c62a2-3647-420e-9ef7-12a41c475b71 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:55.613620 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-8c4c62a2-3647-420e-9ef7-12a41c475b71 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:55.843015 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-8c4c62a2-3647-420e-9ef7-12a41c475b71 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '39de6a6fff7c4cc08cc62aa86182d71d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1235d8b054b3405595fca761f08b73ae', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:44:55.852345 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-8c4c62a2-3647-420e-9ef7-12a41c475b71 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] 10.222.0.26 "GET /compute/v2.1/servers/c7dd981d-4382-47dc-8d9c-effae6f630f3" status: 200 len: 1318 microversion: 2.1 time: 0.264946 Oct 19 00:44:55.853033 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 417/532] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:44:55 2022] GET /compute/v2.1/servers/c7dd981d-4382-47dc-8d9c-effae6f630f3 => generated 1318 bytes in 268 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:44:55.893796 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-fd1d235b-dbb0-402e-b842-e3d6f72b0d0f tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:44:55.894417 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-fd1d235b-dbb0-402e-b842-e3d6f72b0d0f tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] [instance: f6051269-1d16-42db-8323-67a532dbcf40] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:44:55.901776 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-fd1d235b-dbb0-402e-b842-e3d6f72b0d0f tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:55.902272 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-fd1d235b-dbb0-402e-b842-e3d6f72b0d0f tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:55.902753 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-fd1d235b-dbb0-402e-b842-e3d6f72b0d0f tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:55.903489 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-fd1d235b-dbb0-402e-b842-e3d6f72b0d0f tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:55.903927 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-fd1d235b-dbb0-402e-b842-e3d6f72b0d0f tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:55.904348 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-fd1d235b-dbb0-402e-b842-e3d6f72b0d0f tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:56.055693 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-fd1d235b-dbb0-402e-b842-e3d6f72b0d0f tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd3f0d615187d4779bdd7da8c9f24c796', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0151d58abd047a692f5cc4e7d05bea5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:44:56.066739 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-fd1d235b-dbb0-402e-b842-e3d6f72b0d0f tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] 10.222.0.26 "GET /compute/v2.1/servers/f6051269-1d16-42db-8323-67a532dbcf40" status: 200 len: 1725 microversion: 2.1 time: 0.177100 Oct 19 00:44:56.067844 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 418/533] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:44:55 2022] GET /compute/v2.1/servers/f6051269-1d16-42db-8323-67a532dbcf40 => generated 1725 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 19 00:44:56.292426 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [req-249e603f-3502-466a-a01a-247e82cd0f16 req-a8eeb579-0409-4454-b5d0-20d60d891aec service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "c7dd981d-4382-47dc-8d9c-effae6f630f3", "tag": "7c972af5-e79c-417f-bd49-ea410c70f201"}]} {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 19 00:44:56.303616 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-249e603f-3502-466a-a01a-247e82cd0f16 req-a8eeb579-0409-4454-b5d0-20d60d891aec service nova] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:56.304107 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-249e603f-3502-466a-a01a-247e82cd0f16 req-a8eeb579-0409-4454-b5d0-20d60d891aec service nova] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:56.304550 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-249e603f-3502-466a-a01a-247e82cd0f16 req-a8eeb579-0409-4454-b5d0-20d60d891aec service nova] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:56.317366 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.compute.server_external_events [req-249e603f-3502-466a-a01a-247e82cd0f16 req-a8eeb579-0409-4454-b5d0-20d60d891aec service nova] Creating event network-changed:7c972af5-e79c-417f-bd49-ea410c70f201 for instance c7dd981d-4382-47dc-8d9c-effae6f630f3 on hv-cinder-maste Oct 19 00:44:56.323014 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [req-249e603f-3502-466a-a01a-247e82cd0f16 req-a8eeb579-0409-4454-b5d0-20d60d891aec service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.034986 Oct 19 00:44:56.323638 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 419/534] 10.222.0.26 () {64 vars in 1317 bytes} [Wed Oct 19 00:44:56 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 19 00:44:56.617012 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-f996e7b2-a45e-497d-a13c-3b8c7c881f2b tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:44:56.618009 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-f996e7b2-a45e-497d-a13c-3b8c7c881f2b tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] [instance: 0b7c19ee-6d89-4855-bf4b-d35b65bb170b] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:44:56.636378 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-f996e7b2-a45e-497d-a13c-3b8c7c881f2b tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:56.637035 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-f996e7b2-a45e-497d-a13c-3b8c7c881f2b tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:56.637820 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-f996e7b2-a45e-497d-a13c-3b8c7c881f2b tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:56.639154 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-f996e7b2-a45e-497d-a13c-3b8c7c881f2b tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:56.640004 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-f996e7b2-a45e-497d-a13c-3b8c7c881f2b tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:56.640760 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-f996e7b2-a45e-497d-a13c-3b8c7c881f2b tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:56.878452 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-f996e7b2-a45e-497d-a13c-3b8c7c881f2b tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f738e67d8c2340028da1254b60d7810e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8684504b0eff415b9063cebae21ca667', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:44:56.891022 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-f996e7b2-a45e-497d-a13c-3b8c7c881f2b tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] 10.222.0.26 "GET /compute/v2.1/servers/0b7c19ee-6d89-4855-bf4b-d35b65bb170b" status: 200 len: 1266 microversion: 2.1 time: 0.283617 Oct 19 00:44:56.891939 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 420/535] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:44:56 2022] GET /compute/v2.1/servers/0b7c19ee-6d89-4855-bf4b-d35b65bb170b => generated 1266 bytes in 286 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:44:56.898397 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-4b24c76e-5a24-4293-9cb3-5aa1f1c56aab tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:44:56.899413 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-4b24c76e-5a24-4293-9cb3-5aa1f1c56aab tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] [instance: c7dd981d-4382-47dc-8d9c-effae6f630f3] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:44:56.907171 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-4b24c76e-5a24-4293-9cb3-5aa1f1c56aab tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:56.907913 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-4b24c76e-5a24-4293-9cb3-5aa1f1c56aab tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:56.908498 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-4b24c76e-5a24-4293-9cb3-5aa1f1c56aab tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:56.909379 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-4b24c76e-5a24-4293-9cb3-5aa1f1c56aab tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:56.910097 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-4b24c76e-5a24-4293-9cb3-5aa1f1c56aab tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:56.910833 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-4b24c76e-5a24-4293-9cb3-5aa1f1c56aab tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:57.076852 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-4b24c76e-5a24-4293-9cb3-5aa1f1c56aab tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '39de6a6fff7c4cc08cc62aa86182d71d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1235d8b054b3405595fca761f08b73ae', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:44:57.085862 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-4b24c76e-5a24-4293-9cb3-5aa1f1c56aab tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] 10.222.0.26 "GET /compute/v2.1/servers/c7dd981d-4382-47dc-8d9c-effae6f630f3" status: 200 len: 1318 microversion: 2.1 time: 0.192618 Oct 19 00:44:57.086563 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 421/536] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:44:56 2022] GET /compute/v2.1/servers/c7dd981d-4382-47dc-8d9c-effae6f630f3 => generated 1318 bytes in 194 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:44:57.093415 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-b9542fe5-097b-4120-b68d-f6e6afe1e28d tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:44:57.094135 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-b9542fe5-097b-4120-b68d-f6e6afe1e28d tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] [instance: f6051269-1d16-42db-8323-67a532dbcf40] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:44:57.103752 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-b9542fe5-097b-4120-b68d-f6e6afe1e28d tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:57.104571 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-b9542fe5-097b-4120-b68d-f6e6afe1e28d tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:57.105396 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-b9542fe5-097b-4120-b68d-f6e6afe1e28d tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:57.106697 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-b9542fe5-097b-4120-b68d-f6e6afe1e28d tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:57.107561 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-b9542fe5-097b-4120-b68d-f6e6afe1e28d tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:57.108335 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-b9542fe5-097b-4120-b68d-f6e6afe1e28d tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:57.294445 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-b9542fe5-097b-4120-b68d-f6e6afe1e28d tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd3f0d615187d4779bdd7da8c9f24c796', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0151d58abd047a692f5cc4e7d05bea5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:44:57.313067 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-b9542fe5-097b-4120-b68d-f6e6afe1e28d tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] 10.222.0.26 "GET /compute/v2.1/servers/f6051269-1d16-42db-8323-67a532dbcf40" status: 200 len: 1731 microversion: 2.1 time: 0.223545 Oct 19 00:44:57.317750 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 422/537] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:44:57 2022] GET /compute/v2.1/servers/f6051269-1d16-42db-8323-67a532dbcf40 => generated 1731 bytes in 227 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:44:57.357856 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-e242e346-f162-42a5-964f-cec6561cc6c8 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:44:57.358685 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-e242e346-f162-42a5-964f-cec6561cc6c8 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] [instance: f6051269-1d16-42db-8323-67a532dbcf40] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:44:57.370393 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-e242e346-f162-42a5-964f-cec6561cc6c8 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:57.371324 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-e242e346-f162-42a5-964f-cec6561cc6c8 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:57.372110 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-e242e346-f162-42a5-964f-cec6561cc6c8 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:57.373414 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-e242e346-f162-42a5-964f-cec6561cc6c8 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:57.374265 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-e242e346-f162-42a5-964f-cec6561cc6c8 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:57.375090 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-e242e346-f162-42a5-964f-cec6561cc6c8 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:57.485077 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-e242e346-f162-42a5-964f-cec6561cc6c8 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] 10.222.0.26 "GET /compute/v2.1/servers/f6051269-1d16-42db-8323-67a532dbcf40/os-interface" status: 200 len: 285 microversion: 2.1 time: 0.135374 Oct 19 00:44:57.485711 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 423/538] 10.222.0.26 () {58 vars in 1275 bytes} [Wed Oct 19 00:44:57 2022] GET /compute/v2.1/servers/f6051269-1d16-42db-8323-67a532dbcf40/os-interface => generated 285 bytes in 137 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 19 00:44:57.924913 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-50c5abb4-e143-4ce2-a986-ccb7ecb73735 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:44:57.926336 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-50c5abb4-e143-4ce2-a986-ccb7ecb73735 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] [instance: 0b7c19ee-6d89-4855-bf4b-d35b65bb170b] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:44:57.938953 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-50c5abb4-e143-4ce2-a986-ccb7ecb73735 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:57.939614 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-50c5abb4-e143-4ce2-a986-ccb7ecb73735 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:57.940329 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-50c5abb4-e143-4ce2-a986-ccb7ecb73735 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:57.941131 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-50c5abb4-e143-4ce2-a986-ccb7ecb73735 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:57.941700 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-50c5abb4-e143-4ce2-a986-ccb7ecb73735 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:57.942168 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-50c5abb4-e143-4ce2-a986-ccb7ecb73735 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:58.439472 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-50c5abb4-e143-4ce2-a986-ccb7ecb73735 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f738e67d8c2340028da1254b60d7810e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8684504b0eff415b9063cebae21ca667', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:44:58.448954 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-50c5abb4-e143-4ce2-a986-ccb7ecb73735 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] 10.222.0.26 "GET /compute/v2.1/servers/0b7c19ee-6d89-4855-bf4b-d35b65bb170b" status: 200 len: 1266 microversion: 2.1 time: 0.531164 Oct 19 00:44:58.449862 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 424/539] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:44:57 2022] GET /compute/v2.1/servers/0b7c19ee-6d89-4855-bf4b-d35b65bb170b => generated 1266 bytes in 534 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 19 00:44:58.454789 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-1eb9ec3f-bed1-4e01-9d4c-9c7d882b1001 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:44:58.455480 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-1eb9ec3f-bed1-4e01-9d4c-9c7d882b1001 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] [instance: c7dd981d-4382-47dc-8d9c-effae6f630f3] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:44:58.463323 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-1eb9ec3f-bed1-4e01-9d4c-9c7d882b1001 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:58.463894 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-1eb9ec3f-bed1-4e01-9d4c-9c7d882b1001 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:58.464330 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-1eb9ec3f-bed1-4e01-9d4c-9c7d882b1001 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:58.465042 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-1eb9ec3f-bed1-4e01-9d4c-9c7d882b1001 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:58.465493 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-1eb9ec3f-bed1-4e01-9d4c-9c7d882b1001 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:58.465915 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-1eb9ec3f-bed1-4e01-9d4c-9c7d882b1001 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:58.747843 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-1eb9ec3f-bed1-4e01-9d4c-9c7d882b1001 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '39de6a6fff7c4cc08cc62aa86182d71d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1235d8b054b3405595fca761f08b73ae', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:44:58.758545 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-1eb9ec3f-bed1-4e01-9d4c-9c7d882b1001 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] 10.222.0.26 "GET /compute/v2.1/servers/c7dd981d-4382-47dc-8d9c-effae6f630f3" status: 200 len: 1318 microversion: 2.1 time: 0.307526 Oct 19 00:44:58.759359 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 425/540] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:44:58 2022] GET /compute/v2.1/servers/c7dd981d-4382-47dc-8d9c-effae6f630f3 => generated 1318 bytes in 309 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:44:59.004624 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [req-e641d0d4-870b-4106-a826-9ac3a6bd9139 req-c94ac9e3-24be-418d-b6f3-43cf6e782d85 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "f6051269-1d16-42db-8323-67a532dbcf40", "tag": "667c66be-7422-49b6-8e96-6c36f067b697"}]} {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 19 00:44:59.030979 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-e641d0d4-870b-4106-a826-9ac3a6bd9139 req-c94ac9e3-24be-418d-b6f3-43cf6e782d85 service nova] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:59.031972 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-e641d0d4-870b-4106-a826-9ac3a6bd9139 req-c94ac9e3-24be-418d-b6f3-43cf6e782d85 service nova] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:59.032745 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-e641d0d4-870b-4106-a826-9ac3a6bd9139 req-c94ac9e3-24be-418d-b6f3-43cf6e782d85 service nova] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:59.051691 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.compute.server_external_events [req-e641d0d4-870b-4106-a826-9ac3a6bd9139 req-c94ac9e3-24be-418d-b6f3-43cf6e782d85 service nova] Creating event network-changed:667c66be-7422-49b6-8e96-6c36f067b697 for instance f6051269-1d16-42db-8323-67a532dbcf40 on hv-cinder-maste Oct 19 00:44:59.057714 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [req-e641d0d4-870b-4106-a826-9ac3a6bd9139 req-c94ac9e3-24be-418d-b6f3-43cf6e782d85 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.063552 Oct 19 00:44:59.058643 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 426/541] 10.222.0.26 () {64 vars in 1317 bytes} [Wed Oct 19 00:44:58 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 66 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 19 00:44:59.484220 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-90cfc566-b606-436a-942b-887a325e2968 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:44:59.485408 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-90cfc566-b606-436a-942b-887a325e2968 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] [instance: 0b7c19ee-6d89-4855-bf4b-d35b65bb170b] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:44:59.495235 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-90cfc566-b606-436a-942b-887a325e2968 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:59.497413 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-90cfc566-b606-436a-942b-887a325e2968 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:59.500771 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-90cfc566-b606-436a-942b-887a325e2968 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:59.500771 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-90cfc566-b606-436a-942b-887a325e2968 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:59.500771 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-90cfc566-b606-436a-942b-887a325e2968 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:59.501457 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-90cfc566-b606-436a-942b-887a325e2968 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:59.739817 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-90cfc566-b606-436a-942b-887a325e2968 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f738e67d8c2340028da1254b60d7810e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8684504b0eff415b9063cebae21ca667', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:44:59.758243 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-90cfc566-b606-436a-942b-887a325e2968 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] 10.222.0.26 "GET /compute/v2.1/servers/0b7c19ee-6d89-4855-bf4b-d35b65bb170b" status: 200 len: 1453 microversion: 2.1 time: 0.281777 Oct 19 00:44:59.762644 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 427/542] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:44:59 2022] GET /compute/v2.1/servers/0b7c19ee-6d89-4855-bf4b-d35b65bb170b => generated 1453 bytes in 286 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 19 00:44:59.783151 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-f208652e-dd94-4cf0-a7ef-a5273c00d38b tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:44:59.783775 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-f208652e-dd94-4cf0-a7ef-a5273c00d38b tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] [instance: c7dd981d-4382-47dc-8d9c-effae6f630f3] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:44:59.790208 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-f208652e-dd94-4cf0-a7ef-a5273c00d38b tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:59.791907 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-f208652e-dd94-4cf0-a7ef-a5273c00d38b tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:59.792395 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-f208652e-dd94-4cf0-a7ef-a5273c00d38b tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:59.793378 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-f208652e-dd94-4cf0-a7ef-a5273c00d38b tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:44:59.793794 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-f208652e-dd94-4cf0-a7ef-a5273c00d38b tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:44:59.794299 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-f208652e-dd94-4cf0-a7ef-a5273c00d38b tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:44:59.987888 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-f208652e-dd94-4cf0-a7ef-a5273c00d38b tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '39de6a6fff7c4cc08cc62aa86182d71d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1235d8b054b3405595fca761f08b73ae', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:45:00.003900 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-f208652e-dd94-4cf0-a7ef-a5273c00d38b tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] 10.222.0.26 "GET /compute/v2.1/servers/c7dd981d-4382-47dc-8d9c-effae6f630f3" status: 200 len: 1318 microversion: 2.1 time: 0.224227 Oct 19 00:45:00.005695 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 428/543] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:44:59 2022] GET /compute/v2.1/servers/c7dd981d-4382-47dc-8d9c-effae6f630f3 => generated 1318 bytes in 227 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:45:00.015903 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-7c1fdeb8-d49b-414b-9ca3-408117552cb4 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:45:00.018000 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-7c1fdeb8-d49b-414b-9ca3-408117552cb4 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] [instance: 0b7c19ee-6d89-4855-bf4b-d35b65bb170b] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:45:00.032936 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-7c1fdeb8-d49b-414b-9ca3-408117552cb4 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:45:00.034375 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-7c1fdeb8-d49b-414b-9ca3-408117552cb4 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:45:00.035504 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-7c1fdeb8-d49b-414b-9ca3-408117552cb4 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:45:00.037144 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-7c1fdeb8-d49b-414b-9ca3-408117552cb4 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:45:00.037758 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-7c1fdeb8-d49b-414b-9ca3-408117552cb4 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:45:00.038811 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-7c1fdeb8-d49b-414b-9ca3-408117552cb4 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:45:00.208917 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-7c1fdeb8-d49b-414b-9ca3-408117552cb4 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f738e67d8c2340028da1254b60d7810e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8684504b0eff415b9063cebae21ca667', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:45:00.221031 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-7c1fdeb8-d49b-414b-9ca3-408117552cb4 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] 10.222.0.26 "GET /compute/v2.1/servers/0b7c19ee-6d89-4855-bf4b-d35b65bb170b" status: 200 len: 1453 microversion: 2.1 time: 0.213182 Oct 19 00:45:00.221685 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 429/544] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:45:00 2022] GET /compute/v2.1/servers/0b7c19ee-6d89-4855-bf4b-d35b65bb170b => generated 1453 bytes in 215 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:45:00.471276 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-b7bf6499-19ad-40c1-9a3e-e55b4344e6ab tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:45:00.471883 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-b7bf6499-19ad-40c1-9a3e-e55b4344e6ab tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] [instance: b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:45:00.479402 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-b7bf6499-19ad-40c1-9a3e-e55b4344e6ab tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:45:00.479931 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-b7bf6499-19ad-40c1-9a3e-e55b4344e6ab tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:45:00.480383 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-b7bf6499-19ad-40c1-9a3e-e55b4344e6ab tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:45:00.481127 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-b7bf6499-19ad-40c1-9a3e-e55b4344e6ab tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:45:00.481628 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-b7bf6499-19ad-40c1-9a3e-e55b4344e6ab tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:45:00.482083 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-b7bf6499-19ad-40c1-9a3e-e55b4344e6ab tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:45:00.520373 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-b7bf6499-19ad-40c1-9a3e-e55b4344e6ab tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] 10.222.0.26 "GET /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2/os-volume_attachments/b8d00552-fcdb-4fc2-b489-3a2cb14e89e7" status: 200 len: 194 microversion: 2.20 time: 0.053344 Oct 19 00:45:00.521249 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 430/545] 10.222.0.26 () {60 vars in 1454 bytes} [Wed Oct 19 00:45:00 2022] GET /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2/os-volume_attachments/b8d00552-fcdb-4fc2-b489-3a2cb14e89e7 => generated 194 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Oct 19 00:45:00.617181 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-26f6b26a-3424-47e9-8055-dbd6fa8b0321 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:45:00.618061 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-26f6b26a-3424-47e9-8055-dbd6fa8b0321 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] [instance: b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:45:00.629475 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-26f6b26a-3424-47e9-8055-dbd6fa8b0321 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:45:00.630347 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-26f6b26a-3424-47e9-8055-dbd6fa8b0321 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:45:00.631187 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-26f6b26a-3424-47e9-8055-dbd6fa8b0321 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:45:00.632485 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-26f6b26a-3424-47e9-8055-dbd6fa8b0321 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:45:00.638992 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-26f6b26a-3424-47e9-8055-dbd6fa8b0321 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:45:00.640243 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-26f6b26a-3424-47e9-8055-dbd6fa8b0321 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:45:00.736413 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG dbcounter [-] [125641] Writing DB stats nova_cell1:SELECT=100 {{(pid=125641) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 00:45:00.741786 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.objects.instance [None req-26f6b26a-3424-47e9-8055-dbd6fa8b0321 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lazy-loading 'services' on Instance uuid b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2 {{(pid=125641) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 19 00:45:00.827564 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-26f6b26a-3424-47e9-8055-dbd6fa8b0321 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] 10.222.0.26 "DELETE /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2/os-volume_attachments/b8d00552-fcdb-4fc2-b489-3a2cb14e89e7" status: 202 len: 0 microversion: 2.20 time: 0.217880 Oct 19 00:45:00.828212 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 431/546] 10.222.0.26 () {60 vars in 1457 bytes} [Wed Oct 19 00:45:00 2022] DELETE /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2/os-volume_attachments/b8d00552-fcdb-4fc2-b489-3a2cb14e89e7 => generated 0 bytes in 220 msecs (HTTP/1.1 202) 9 headers in 363 bytes (1 switches on core 0) Oct 19 00:45:00.838328 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-687a3a34-7788-4ff3-a8b5-9edade9be63c tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:45:00.840819 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-687a3a34-7788-4ff3-a8b5-9edade9be63c tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] [instance: b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:45:00.850087 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-687a3a34-7788-4ff3-a8b5-9edade9be63c tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:45:00.850972 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-687a3a34-7788-4ff3-a8b5-9edade9be63c tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:45:00.851770 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-687a3a34-7788-4ff3-a8b5-9edade9be63c tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:45:00.853787 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-687a3a34-7788-4ff3-a8b5-9edade9be63c tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:45:00.854619 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-687a3a34-7788-4ff3-a8b5-9edade9be63c tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:45:00.855407 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-687a3a34-7788-4ff3-a8b5-9edade9be63c tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:45:00.891813 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-687a3a34-7788-4ff3-a8b5-9edade9be63c tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] 10.222.0.26 "GET /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.058532 Oct 19 00:45:00.892410 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 432/547] 10.222.0.26 () {60 vars in 1343 bytes} [Wed Oct 19 00:45:00 2022] GET /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2/os-volume_attachments => generated 197 bytes in 60 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Oct 19 00:45:01.053181 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-d423f20d-28ba-41b9-97e6-89c3e457ea0e tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:45:01.055148 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-d423f20d-28ba-41b9-97e6-89c3e457ea0e tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] [instance: c7dd981d-4382-47dc-8d9c-effae6f630f3] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:45:01.066673 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d423f20d-28ba-41b9-97e6-89c3e457ea0e tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:45:01.067438 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d423f20d-28ba-41b9-97e6-89c3e457ea0e tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:45:01.068181 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d423f20d-28ba-41b9-97e6-89c3e457ea0e tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:45:01.069507 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d423f20d-28ba-41b9-97e6-89c3e457ea0e tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:45:01.070307 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d423f20d-28ba-41b9-97e6-89c3e457ea0e tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:45:01.071062 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d423f20d-28ba-41b9-97e6-89c3e457ea0e tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:45:01.249551 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-d423f20d-28ba-41b9-97e6-89c3e457ea0e tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '39de6a6fff7c4cc08cc62aa86182d71d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1235d8b054b3405595fca761f08b73ae', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:45:01.260088 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-d423f20d-28ba-41b9-97e6-89c3e457ea0e tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] 10.222.0.26 "GET /compute/v2.1/servers/c7dd981d-4382-47dc-8d9c-effae6f630f3" status: 200 len: 1318 microversion: 2.1 time: 0.210865 Oct 19 00:45:01.260843 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 433/548] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:45:01 2022] GET /compute/v2.1/servers/c7dd981d-4382-47dc-8d9c-effae6f630f3 => generated 1318 bytes in 213 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:45:01.272467 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [req-0f58afa9-0400-4dc8-91d6-ec6060efd351 req-c35301cf-b610-451e-a472-440e0ec2c28e service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "f6051269-1d16-42db-8323-67a532dbcf40", "tag": "667c66be-7422-49b6-8e96-6c36f067b697"}]} {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 19 00:45:01.284201 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-0f58afa9-0400-4dc8-91d6-ec6060efd351 req-c35301cf-b610-451e-a472-440e0ec2c28e service nova] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:45:01.284681 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-0f58afa9-0400-4dc8-91d6-ec6060efd351 req-c35301cf-b610-451e-a472-440e0ec2c28e service nova] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:45:01.285114 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-0f58afa9-0400-4dc8-91d6-ec6060efd351 req-c35301cf-b610-451e-a472-440e0ec2c28e service nova] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:45:01.297981 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.compute.server_external_events [req-0f58afa9-0400-4dc8-91d6-ec6060efd351 req-c35301cf-b610-451e-a472-440e0ec2c28e service nova] Creating event network-changed:667c66be-7422-49b6-8e96-6c36f067b697 for instance f6051269-1d16-42db-8323-67a532dbcf40 on hv-cinder-maste Oct 19 00:45:01.303739 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [req-0f58afa9-0400-4dc8-91d6-ec6060efd351 req-c35301cf-b610-451e-a472-440e0ec2c28e service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.035060 Oct 19 00:45:01.304546 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 434/549] 10.222.0.26 () {64 vars in 1317 bytes} [Wed Oct 19 00:45:01 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 19 00:45:01.907827 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-468bd976-a606-4aee-b36e-2bf8d735839f tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:45:01.910513 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-468bd976-a606-4aee-b36e-2bf8d735839f tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] [instance: b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:45:01.919100 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-468bd976-a606-4aee-b36e-2bf8d735839f tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:45:01.919716 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-468bd976-a606-4aee-b36e-2bf8d735839f tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:45:01.920509 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-468bd976-a606-4aee-b36e-2bf8d735839f tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:45:01.921359 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-468bd976-a606-4aee-b36e-2bf8d735839f tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:45:01.921967 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-468bd976-a606-4aee-b36e-2bf8d735839f tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:45:01.922618 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-468bd976-a606-4aee-b36e-2bf8d735839f tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:45:01.958433 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-468bd976-a606-4aee-b36e-2bf8d735839f tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] 10.222.0.26 "GET /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.055988 Oct 19 00:45:01.959062 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 435/550] 10.222.0.26 () {60 vars in 1343 bytes} [Wed Oct 19 00:45:01 2022] GET /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2/os-volume_attachments => generated 197 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Oct 19 00:45:02.284173 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-fb7ab1ea-58ac-430c-b85e-b120b36053e8 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:45:02.284853 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-fb7ab1ea-58ac-430c-b85e-b120b36053e8 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] [instance: c7dd981d-4382-47dc-8d9c-effae6f630f3] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:45:02.287883 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG dbcounter [-] [125641] Writing DB stats nova_api:SELECT=100 {{(pid=125641) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 00:45:02.294007 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-fb7ab1ea-58ac-430c-b85e-b120b36053e8 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:45:02.294582 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-fb7ab1ea-58ac-430c-b85e-b120b36053e8 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:45:02.294980 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-fb7ab1ea-58ac-430c-b85e-b120b36053e8 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:45:02.295702 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-fb7ab1ea-58ac-430c-b85e-b120b36053e8 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:45:02.296202 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-fb7ab1ea-58ac-430c-b85e-b120b36053e8 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:45:02.296636 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-fb7ab1ea-58ac-430c-b85e-b120b36053e8 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:45:02.454882 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-fb7ab1ea-58ac-430c-b85e-b120b36053e8 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '39de6a6fff7c4cc08cc62aa86182d71d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1235d8b054b3405595fca761f08b73ae', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:45:02.463621 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-fb7ab1ea-58ac-430c-b85e-b120b36053e8 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] 10.222.0.26 "GET /compute/v2.1/servers/c7dd981d-4382-47dc-8d9c-effae6f630f3" status: 200 len: 1318 microversion: 2.1 time: 0.183277 Oct 19 00:45:02.478532 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 436/551] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:45:02 2022] GET /compute/v2.1/servers/c7dd981d-4382-47dc-8d9c-effae6f630f3 => generated 1318 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 19 00:45:02.972452 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-076f20e3-40b5-4b7f-841f-5c81a6f945b4 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:45:02.974772 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-076f20e3-40b5-4b7f-841f-5c81a6f945b4 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] [instance: b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:45:02.982740 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-076f20e3-40b5-4b7f-841f-5c81a6f945b4 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:45:02.983147 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-076f20e3-40b5-4b7f-841f-5c81a6f945b4 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:45:02.983588 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-076f20e3-40b5-4b7f-841f-5c81a6f945b4 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:45:02.984341 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-076f20e3-40b5-4b7f-841f-5c81a6f945b4 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:45:02.984780 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-076f20e3-40b5-4b7f-841f-5c81a6f945b4 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:45:02.985205 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-076f20e3-40b5-4b7f-841f-5c81a6f945b4 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:45:03.018874 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-076f20e3-40b5-4b7f-841f-5c81a6f945b4 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] 10.222.0.26 "GET /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2/os-volume_attachments" status: 200 len: 25 microversion: 2.20 time: 0.050865 Oct 19 00:45:03.019544 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 437/552] 10.222.0.26 () {60 vars in 1343 bytes} [Wed Oct 19 00:45:02 2022] GET /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2/os-volume_attachments => generated 25 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 19 00:45:03.096554 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-f725f182-16f3-448a-9ab8-20d400838ccb tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:45:03.097439 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-f725f182-16f3-448a-9ab8-20d400838ccb tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] [instance: b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:45:03.108591 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-f725f182-16f3-448a-9ab8-20d400838ccb tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:45:03.109509 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-f725f182-16f3-448a-9ab8-20d400838ccb tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:45:03.110472 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-f725f182-16f3-448a-9ab8-20d400838ccb tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:45:03.112333 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-f725f182-16f3-448a-9ab8-20d400838ccb tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:45:03.113338 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-f725f182-16f3-448a-9ab8-20d400838ccb tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:45:03.114228 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-f725f182-16f3-448a-9ab8-20d400838ccb tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:45:03.167325 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-f725f182-16f3-448a-9ab8-20d400838ccb tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] [instance: b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2] Going to try to terminate instance {{(pid=125641) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 19 00:45:03.314966 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-f725f182-16f3-448a-9ab8-20d400838ccb tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] 10.222.0.26 "DELETE /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2" status: 204 len: 0 microversion: 2.20 time: 0.223128 Oct 19 00:45:03.316314 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 438/553] 10.222.0.26 () {60 vars in 1280 bytes} [Wed Oct 19 00:45:03 2022] DELETE /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 9 headers in 365 bytes (2 switches on core 0) Oct 19 00:45:03.325504 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-a7ee168a-c660-4b42-8a31-d0b848b73e7b tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:45:03.326219 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-a7ee168a-c660-4b42-8a31-d0b848b73e7b tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] [instance: b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:45:03.333306 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-a7ee168a-c660-4b42-8a31-d0b848b73e7b tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:45:03.334197 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-a7ee168a-c660-4b42-8a31-d0b848b73e7b tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:45:03.334713 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-a7ee168a-c660-4b42-8a31-d0b848b73e7b tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:45:03.335459 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-a7ee168a-c660-4b42-8a31-d0b848b73e7b tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:45:03.336118 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-a7ee168a-c660-4b42-8a31-d0b848b73e7b tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:45:03.336555 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-a7ee168a-c660-4b42-8a31-d0b848b73e7b tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:45:03.521879 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-a7ee168a-c660-4b42-8a31-d0b848b73e7b tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:45:03.539467 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-a7ee168a-c660-4b42-8a31-d0b848b73e7b tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:45:03.543271 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-a7ee168a-c660-4b42-8a31-d0b848b73e7b tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:45:03.544915 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-a7ee168a-c660-4b42-8a31-d0b848b73e7b tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] 10.222.0.26 "GET /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2" status: 200 len: 1723 microversion: 2.20 time: 0.223534 Oct 19 00:45:03.549771 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 439/554] 10.222.0.26 () {60 vars in 1277 bytes} [Wed Oct 19 00:45:03 2022] GET /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2 => generated 1723 bytes in 226 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 19 00:45:03.554936 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-051ae9c3-90fb-4020-8c11-1ad9d48820bf tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:45:03.556572 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-051ae9c3-90fb-4020-8c11-1ad9d48820bf tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] [instance: c7dd981d-4382-47dc-8d9c-effae6f630f3] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:45:03.567602 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-051ae9c3-90fb-4020-8c11-1ad9d48820bf tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:45:03.568478 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-051ae9c3-90fb-4020-8c11-1ad9d48820bf tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:45:03.569266 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-051ae9c3-90fb-4020-8c11-1ad9d48820bf tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:45:03.570552 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-051ae9c3-90fb-4020-8c11-1ad9d48820bf tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:45:03.572031 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-051ae9c3-90fb-4020-8c11-1ad9d48820bf tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:45:03.573503 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-051ae9c3-90fb-4020-8c11-1ad9d48820bf tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:45:03.787504 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-051ae9c3-90fb-4020-8c11-1ad9d48820bf tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '39de6a6fff7c4cc08cc62aa86182d71d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1235d8b054b3405595fca761f08b73ae', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:45:03.797710 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-051ae9c3-90fb-4020-8c11-1ad9d48820bf tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] 10.222.0.26 "GET /compute/v2.1/servers/c7dd981d-4382-47dc-8d9c-effae6f630f3" status: 200 len: 1318 microversion: 2.1 time: 0.249840 Oct 19 00:45:03.798476 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 440/555] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:45:03 2022] GET /compute/v2.1/servers/c7dd981d-4382-47dc-8d9c-effae6f630f3 => generated 1318 bytes in 252 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:45:03.920309 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [req-b792bf3f-b278-43aa-a6c4-c74100e68928 req-c36434fa-04c2-4d68-afe5-f2a000a10fb6 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "0b7c19ee-6d89-4855-bf4b-d35b65bb170b", "tag": "3f92cbb3-bb62-457d-b0d7-92947884c5fb"}]} {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 19 00:45:03.931072 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-b792bf3f-b278-43aa-a6c4-c74100e68928 req-c36434fa-04c2-4d68-afe5-f2a000a10fb6 service nova] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:45:03.931663 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-b792bf3f-b278-43aa-a6c4-c74100e68928 req-c36434fa-04c2-4d68-afe5-f2a000a10fb6 service nova] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:45:03.932171 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-b792bf3f-b278-43aa-a6c4-c74100e68928 req-c36434fa-04c2-4d68-afe5-f2a000a10fb6 service nova] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:45:03.945858 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.compute.server_external_events [req-b792bf3f-b278-43aa-a6c4-c74100e68928 req-c36434fa-04c2-4d68-afe5-f2a000a10fb6 service nova] Creating event network-changed:3f92cbb3-bb62-457d-b0d7-92947884c5fb for instance 0b7c19ee-6d89-4855-bf4b-d35b65bb170b on hv-cinder-maste Oct 19 00:45:03.952595 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [req-b792bf3f-b278-43aa-a6c4-c74100e68928 req-c36434fa-04c2-4d68-afe5-f2a000a10fb6 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.039044 Oct 19 00:45:03.953487 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 441/556] 10.222.0.26 () {64 vars in 1317 bytes} [Wed Oct 19 00:45:03 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 19 00:45:04.579664 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-4cb1559d-87f6-4cd5-80a0-4c12aedbef58 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:45:04.581551 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-4cb1559d-87f6-4cd5-80a0-4c12aedbef58 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] [instance: b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:45:04.594207 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-4cb1559d-87f6-4cd5-80a0-4c12aedbef58 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:45:04.595816 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-4cb1559d-87f6-4cd5-80a0-4c12aedbef58 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:45:04.596618 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-4cb1559d-87f6-4cd5-80a0-4c12aedbef58 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:45:04.597928 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-4cb1559d-87f6-4cd5-80a0-4c12aedbef58 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:45:04.598738 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-4cb1559d-87f6-4cd5-80a0-4c12aedbef58 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:45:04.599526 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-4cb1559d-87f6-4cd5-80a0-4c12aedbef58 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:45:04.839003 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-4cb1559d-87f6-4cd5-80a0-4c12aedbef58 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:45:04.852604 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-4cb1559d-87f6-4cd5-80a0-4c12aedbef58 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:45:04.855748 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-4cb1559d-87f6-4cd5-80a0-4c12aedbef58 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:45:04.858566 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-4cb1559d-87f6-4cd5-80a0-4c12aedbef58 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] 10.222.0.26 "GET /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2" status: 200 len: 1723 microversion: 2.20 time: 0.288629 Oct 19 00:45:04.859678 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 442/557] 10.222.0.26 () {60 vars in 1277 bytes} [Wed Oct 19 00:45:04 2022] GET /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2 => generated 1723 bytes in 291 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 19 00:45:04.865045 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-24eb752d-3ab3-4020-98aa-7c2b11b3783c tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:45:04.865676 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-24eb752d-3ab3-4020-98aa-7c2b11b3783c tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] [instance: c7dd981d-4382-47dc-8d9c-effae6f630f3] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:45:04.873875 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-24eb752d-3ab3-4020-98aa-7c2b11b3783c tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:45:04.874526 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-24eb752d-3ab3-4020-98aa-7c2b11b3783c tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:45:04.874968 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-24eb752d-3ab3-4020-98aa-7c2b11b3783c tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:45:04.875736 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-24eb752d-3ab3-4020-98aa-7c2b11b3783c tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:45:04.876433 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-24eb752d-3ab3-4020-98aa-7c2b11b3783c tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:45:04.876874 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-24eb752d-3ab3-4020-98aa-7c2b11b3783c tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:45:05.132858 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-24eb752d-3ab3-4020-98aa-7c2b11b3783c tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '39de6a6fff7c4cc08cc62aa86182d71d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1235d8b054b3405595fca761f08b73ae', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:45:05.142532 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-24eb752d-3ab3-4020-98aa-7c2b11b3783c tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] 10.222.0.26 "GET /compute/v2.1/servers/c7dd981d-4382-47dc-8d9c-effae6f630f3" status: 200 len: 1318 microversion: 2.1 time: 0.281674 Oct 19 00:45:05.147538 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 443/558] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:45:04 2022] GET /compute/v2.1/servers/c7dd981d-4382-47dc-8d9c-effae6f630f3 => generated 1318 bytes in 283 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:45:05.148434 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [req-ddb217eb-8e4d-4526-b0ce-ec32fea6c195 req-1d405513-a092-4ef8-8d88-5a2adb4e4a75 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "c7dd981d-4382-47dc-8d9c-effae6f630f3", "name": "network-vif-plugged", "status": "completed", "tag": "7c972af5-e79c-417f-bd49-ea410c70f201"}]} {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 19 00:45:05.159449 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-ddb217eb-8e4d-4526-b0ce-ec32fea6c195 req-1d405513-a092-4ef8-8d88-5a2adb4e4a75 service nova] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:45:05.159967 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-ddb217eb-8e4d-4526-b0ce-ec32fea6c195 req-1d405513-a092-4ef8-8d88-5a2adb4e4a75 service nova] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:45:05.160447 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-ddb217eb-8e4d-4526-b0ce-ec32fea6c195 req-1d405513-a092-4ef8-8d88-5a2adb4e4a75 service nova] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:45:05.173505 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.compute.server_external_events [req-ddb217eb-8e4d-4526-b0ce-ec32fea6c195 req-1d405513-a092-4ef8-8d88-5a2adb4e4a75 service nova] Creating event network-vif-plugged:7c972af5-e79c-417f-bd49-ea410c70f201 for instance c7dd981d-4382-47dc-8d9c-effae6f630f3 on hv-cinder-maste Oct 19 00:45:05.178819 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [req-ddb217eb-8e4d-4526-b0ce-ec32fea6c195 req-1d405513-a092-4ef8-8d88-5a2adb4e4a75 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.034254 Oct 19 00:45:05.179472 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 444/559] 10.222.0.26 () {64 vars in 1317 bytes} [Wed Oct 19 00:45:05 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 19 00:45:05.907004 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-bc0bec0e-43a6-4be5-9b70-a95b9f694c8d tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:45:05.908189 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-bc0bec0e-43a6-4be5-9b70-a95b9f694c8d tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] [instance: b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:45:05.916935 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-bc0bec0e-43a6-4be5-9b70-a95b9f694c8d tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:45:05.917575 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-bc0bec0e-43a6-4be5-9b70-a95b9f694c8d tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:45:05.918240 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-bc0bec0e-43a6-4be5-9b70-a95b9f694c8d tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:45:05.919374 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-bc0bec0e-43a6-4be5-9b70-a95b9f694c8d tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:45:05.919996 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-bc0bec0e-43a6-4be5-9b70-a95b9f694c8d tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:45:05.920712 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-bc0bec0e-43a6-4be5-9b70-a95b9f694c8d tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:45:06.081975 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-bc0bec0e-43a6-4be5-9b70-a95b9f694c8d tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:45:06.097992 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-bc0bec0e-43a6-4be5-9b70-a95b9f694c8d tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:45:06.100317 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-bc0bec0e-43a6-4be5-9b70-a95b9f694c8d tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:45:06.103059 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-bc0bec0e-43a6-4be5-9b70-a95b9f694c8d tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] 10.222.0.26 "GET /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2" status: 200 len: 1723 microversion: 2.20 time: 0.201147 Oct 19 00:45:06.104401 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 445/560] 10.222.0.26 () {60 vars in 1277 bytes} [Wed Oct 19 00:45:05 2022] GET /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2 => generated 1723 bytes in 205 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 19 00:45:06.167666 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-8edae71c-34be-468d-8296-4973f8f799e2 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:45:06.168296 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-8edae71c-34be-468d-8296-4973f8f799e2 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] [instance: c7dd981d-4382-47dc-8d9c-effae6f630f3] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:45:06.176220 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-8edae71c-34be-468d-8296-4973f8f799e2 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:45:06.177068 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-8edae71c-34be-468d-8296-4973f8f799e2 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:45:06.177852 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-8edae71c-34be-468d-8296-4973f8f799e2 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:45:06.178794 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-8edae71c-34be-468d-8296-4973f8f799e2 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:45:06.179355 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-8edae71c-34be-468d-8296-4973f8f799e2 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:45:06.180449 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-8edae71c-34be-468d-8296-4973f8f799e2 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:45:06.397402 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-8edae71c-34be-468d-8296-4973f8f799e2 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '39de6a6fff7c4cc08cc62aa86182d71d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1235d8b054b3405595fca761f08b73ae', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:45:06.418366 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-8edae71c-34be-468d-8296-4973f8f799e2 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] 10.222.0.26 "GET /compute/v2.1/servers/c7dd981d-4382-47dc-8d9c-effae6f630f3" status: 200 len: 1318 microversion: 2.1 time: 0.254660 Oct 19 00:45:06.420196 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 446/561] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:45:06 2022] GET /compute/v2.1/servers/c7dd981d-4382-47dc-8d9c-effae6f630f3 => generated 1318 bytes in 257 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 19 00:45:07.140952 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-8d00e9de-6982-4c90-81b0-f53cc5ced316 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:45:07.142617 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-8d00e9de-6982-4c90-81b0-f53cc5ced316 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] [instance: b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:45:07.151020 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-8d00e9de-6982-4c90-81b0-f53cc5ced316 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:45:07.151817 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-8d00e9de-6982-4c90-81b0-f53cc5ced316 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:45:07.152277 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-8d00e9de-6982-4c90-81b0-f53cc5ced316 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:45:07.153024 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-8d00e9de-6982-4c90-81b0-f53cc5ced316 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:45:07.153484 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-8d00e9de-6982-4c90-81b0-f53cc5ced316 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:45:07.153908 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-8d00e9de-6982-4c90-81b0-f53cc5ced316 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:45:07.252665 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-8d00e9de-6982-4c90-81b0-f53cc5ced316 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:45:07.264378 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-8d00e9de-6982-4c90-81b0-f53cc5ced316 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:45:07.265989 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-8d00e9de-6982-4c90-81b0-f53cc5ced316 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:45:07.267378 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-8d00e9de-6982-4c90-81b0-f53cc5ced316 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] 10.222.0.26 "GET /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2" status: 200 len: 1655 microversion: 2.20 time: 0.131078 Oct 19 00:45:07.282910 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 447/562] 10.222.0.26 () {60 vars in 1277 bytes} [Wed Oct 19 00:45:07 2022] GET /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2 => generated 1655 bytes in 134 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 19 00:45:07.282910 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [req-da70818f-fbb9-422a-a94e-ceb163193712 req-42cfeefd-657d-49c8-bdbb-0d360748c19a service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2", "name": "network-vif-deleted", "tag": "6aea38b4-b7a0-4e8e-82dc-deb19c260092"}]} {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 19 00:45:07.284840 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-da70818f-fbb9-422a-a94e-ceb163193712 req-42cfeefd-657d-49c8-bdbb-0d360748c19a service nova] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:45:07.285331 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-da70818f-fbb9-422a-a94e-ceb163193712 req-42cfeefd-657d-49c8-bdbb-0d360748c19a service nova] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:45:07.286695 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-da70818f-fbb9-422a-a94e-ceb163193712 req-42cfeefd-657d-49c8-bdbb-0d360748c19a service nova] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:45:07.292188 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG dbcounter [-] [125641] Writing DB stats nova_cell1:SELECT=97,nova_cell1:INSERT=2,nova_cell1:UPDATE=1 {{(pid=125641) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 00:45:07.301093 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.compute.server_external_events [req-da70818f-fbb9-422a-a94e-ceb163193712 req-42cfeefd-657d-49c8-bdbb-0d360748c19a service nova] Creating event network-vif-deleted:6aea38b4-b7a0-4e8e-82dc-deb19c260092 for instance b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2 on hv-cinder-maste Oct 19 00:45:07.307332 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [req-da70818f-fbb9-422a-a94e-ceb163193712 req-42cfeefd-657d-49c8-bdbb-0d360748c19a service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.037465 Oct 19 00:45:07.312468 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 448/563] 10.222.0.26 () {64 vars in 1317 bytes} [Wed Oct 19 00:45:07 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 19 00:45:07.454966 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-e3fca457-6eb4-4a76-8cd5-f2d5fbd8e8bf tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:45:07.454966 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-e3fca457-6eb4-4a76-8cd5-f2d5fbd8e8bf tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] [instance: c7dd981d-4382-47dc-8d9c-effae6f630f3] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:45:07.473467 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-e3fca457-6eb4-4a76-8cd5-f2d5fbd8e8bf tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:45:07.473969 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-e3fca457-6eb4-4a76-8cd5-f2d5fbd8e8bf tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:45:07.474569 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-e3fca457-6eb4-4a76-8cd5-f2d5fbd8e8bf tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:45:07.475310 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-e3fca457-6eb4-4a76-8cd5-f2d5fbd8e8bf tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:45:07.475771 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-e3fca457-6eb4-4a76-8cd5-f2d5fbd8e8bf tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:45:07.477179 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-e3fca457-6eb4-4a76-8cd5-f2d5fbd8e8bf tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:45:07.676178 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-e3fca457-6eb4-4a76-8cd5-f2d5fbd8e8bf tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '39de6a6fff7c4cc08cc62aa86182d71d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1235d8b054b3405595fca761f08b73ae', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:45:07.685305 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-e3fca457-6eb4-4a76-8cd5-f2d5fbd8e8bf tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] 10.222.0.26 "GET /compute/v2.1/servers/c7dd981d-4382-47dc-8d9c-effae6f630f3" status: 200 len: 1318 microversion: 2.1 time: 0.237713 Oct 19 00:45:07.686278 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 449/564] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:45:07 2022] GET /compute/v2.1/servers/c7dd981d-4382-47dc-8d9c-effae6f630f3 => generated 1318 bytes in 241 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 19 00:45:08.309248 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-47a2f920-57c4-423a-98a7-f76b339944cb tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:45:08.310544 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-47a2f920-57c4-423a-98a7-f76b339944cb tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] [instance: b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:45:08.322307 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-47a2f920-57c4-423a-98a7-f76b339944cb tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:45:08.323428 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-47a2f920-57c4-423a-98a7-f76b339944cb tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:45:08.324297 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-47a2f920-57c4-423a-98a7-f76b339944cb tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:45:08.325716 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-47a2f920-57c4-423a-98a7-f76b339944cb tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:45:08.326925 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-47a2f920-57c4-423a-98a7-f76b339944cb tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:45:08.327844 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-47a2f920-57c4-423a-98a7-f76b339944cb tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:45:08.394928 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-47a2f920-57c4-423a-98a7-f76b339944cb tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:45:08.395933 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-47a2f920-57c4-423a-98a7-f76b339944cb tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:45:08.396849 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-47a2f920-57c4-423a-98a7-f76b339944cb tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:45:08.397838 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.objects.instance [None req-47a2f920-57c4-423a-98a7-f76b339944cb tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lazy-loading 'fault' on Instance uuid b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2 {{(pid=125641) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 19 00:45:08.458720 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-47a2f920-57c4-423a-98a7-f76b339944cb tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:45:08.472078 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-47a2f920-57c4-423a-98a7-f76b339944cb tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:45:08.474212 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-47a2f920-57c4-423a-98a7-f76b339944cb tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:45:08.475798 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-47a2f920-57c4-423a-98a7-f76b339944cb tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] 10.222.0.26 "GET /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2" status: 200 len: 1371 microversion: 2.20 time: 0.174921 Oct 19 00:45:08.476546 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 450/565] 10.222.0.26 () {60 vars in 1277 bytes} [Wed Oct 19 00:45:08 2022] GET /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2 => generated 1371 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 19 00:45:08.714168 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-922c5b9c-578c-413d-aa7c-afb34172709c tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:45:08.715342 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-922c5b9c-578c-413d-aa7c-afb34172709c tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] [instance: c7dd981d-4382-47dc-8d9c-effae6f630f3] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:45:08.724036 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-922c5b9c-578c-413d-aa7c-afb34172709c tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:45:08.724564 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-922c5b9c-578c-413d-aa7c-afb34172709c tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:45:08.725034 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-922c5b9c-578c-413d-aa7c-afb34172709c tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:45:08.725749 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-922c5b9c-578c-413d-aa7c-afb34172709c tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:45:08.726231 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-922c5b9c-578c-413d-aa7c-afb34172709c tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:45:08.726706 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-922c5b9c-578c-413d-aa7c-afb34172709c tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:45:08.916851 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-922c5b9c-578c-413d-aa7c-afb34172709c tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '39de6a6fff7c4cc08cc62aa86182d71d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1235d8b054b3405595fca761f08b73ae', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:45:08.925566 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-922c5b9c-578c-413d-aa7c-afb34172709c tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] 10.222.0.26 "GET /compute/v2.1/servers/c7dd981d-4382-47dc-8d9c-effae6f630f3" status: 200 len: 1507 microversion: 2.1 time: 0.219404 Oct 19 00:45:08.926086 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 451/566] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:45:08 2022] GET /compute/v2.1/servers/c7dd981d-4382-47dc-8d9c-effae6f630f3 => generated 1507 bytes in 221 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:45:08.948810 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-4f68e1ac-1b4e-4ed3-a93e-242b370b79ea tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "06efee89-ad39-4197-948d-1ef3be836331", "device": "/dev/sdb"}} {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 19 00:45:08.951305 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-4f68e1ac-1b4e-4ed3-a93e-242b370b79ea tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] [instance: c7dd981d-4382-47dc-8d9c-effae6f630f3] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:45:08.957875 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-4f68e1ac-1b4e-4ed3-a93e-242b370b79ea tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:45:08.958465 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-4f68e1ac-1b4e-4ed3-a93e-242b370b79ea tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:45:08.959009 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-4f68e1ac-1b4e-4ed3-a93e-242b370b79ea tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:45:08.959819 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-4f68e1ac-1b4e-4ed3-a93e-242b370b79ea tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:45:08.960331 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-4f68e1ac-1b4e-4ed3-a93e-242b370b79ea tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:45:08.960819 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-4f68e1ac-1b4e-4ed3-a93e-242b370b79ea tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:45:09.272470 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-4f68e1ac-1b4e-4ed3-a93e-242b370b79ea tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] 10.222.0.26 "POST /compute/v2.1/servers/c7dd981d-4382-47dc-8d9c-effae6f630f3/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.327605 Oct 19 00:45:09.273096 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 452/567] 10.222.0.26 () {60 vars in 1323 bytes} [Wed Oct 19 00:45:08 2022] POST /compute/v2.1/servers/c7dd981d-4382-47dc-8d9c-effae6f630f3/os-volume_attachments => generated 194 bytes in 329 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 19 00:45:09.504260 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-56726637-824c-4335-834a-7a8bc41b04a8 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:45:09.505191 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-56726637-824c-4335-834a-7a8bc41b04a8 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] [instance: b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:45:09.515294 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-56726637-824c-4335-834a-7a8bc41b04a8 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:45:09.515865 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-56726637-824c-4335-834a-7a8bc41b04a8 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:45:09.516380 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-56726637-824c-4335-834a-7a8bc41b04a8 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:45:09.517254 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-56726637-824c-4335-834a-7a8bc41b04a8 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:45:09.517802 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-56726637-824c-4335-834a-7a8bc41b04a8 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:45:09.518299 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-56726637-824c-4335-834a-7a8bc41b04a8 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:45:09.536313 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.wsgi [None req-56726637-824c-4335-834a-7a8bc41b04a8 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] HTTP exception thrown: Instance b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2 could not be found. Oct 19 00:45:09.538161 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-56726637-824c-4335-834a-7a8bc41b04a8 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Request method failure captured: Oct 19 00:45:09.538161 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: request: GET /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2 HTTP/1.1 Oct 19 00:45:09.538161 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Accept: application/json Oct 19 00:45:09.538161 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Accept-Encoding: identity Oct 19 00:45:09.538161 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Connection: close Oct 19 00:45:09.538161 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Content-Length: 0 Oct 19 00:45:09.538161 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Content-Type: application/json Oct 19 00:45:09.538161 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Host: 10.222.0.26 Oct 19 00:45:09.538161 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Openstack-System-Scope: None Oct 19 00:45:09.538161 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: User-Agent: python-urllib3/1.26.12 Oct 19 00:45:09.538161 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Auth-Token: gAAAAABjT0eHghZZjQqIuLCd4UKjhyLHbqvZwQOzf8mMbPeH2urCSODAi7cb1nzpcISqPOiEw0ao4lxHIgJNa4VK2pIrrAVhMlfGsoDt_GEPXgOU1OcEl_JfKpXVIRpiEPvnqRQQAWlaMAyPJ6rWW3fskjR5vymFctg8L4gTCyMdOZ7pK8ZqEBg Oct 19 00:45:09.538161 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Domain-Id: None Oct 19 00:45:09.538161 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Domain-Name: None Oct 19 00:45:09.538161 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Identity-Status: Confirmed Oct 19 00:45:09.538161 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Is-Admin-Project: True Oct 19 00:45:09.538161 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Openstack-Nova-Api-Version: 2.20 Oct 19 00:45:09.538161 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Project-Domain-Id: default Oct 19 00:45:09.538161 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Project-Domain-Name: Default Oct 19 00:45:09.538161 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Project-Id: 37ffcffcae9b41bbb2d13adc49cb99df Oct 19 00:45:09.538161 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-495634389 Oct 19 00:45:09.538161 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Role: member,reader Oct 19 00:45:09.538161 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Roles: member,reader Oct 19 00:45:09.540595 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/37ffcffcae9b41bbb2d13adc49cb99df"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/37ffcffcae9b41bbb2d13adc49cb99df"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/37ffcffcae9b41bbb2d13adc49cb99df"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_37ffcffcae9b41bbb2d13adc49cb99df"}]}] Oct 19 00:45:09.540595 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Tenant: tempest-AttachVolumeShelveTestJSON-495634389 Oct 19 00:45:09.540595 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Tenant-Id: 37ffcffcae9b41bbb2d13adc49cb99df Oct 19 00:45:09.540595 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-495634389 Oct 19 00:45:09.540595 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User: tempest-AttachVolumeShelveTestJSON-495634389-project Oct 19 00:45:09.540595 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User-Domain-Id: default Oct 19 00:45:09.540595 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User-Domain-Name: Default Oct 19 00:45:09.540595 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User-Id: 69fef5650fff42e4ac4ed3ca74d4b123 Oct 19 00:45:09.540595 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User-Name: tempest-AttachVolumeShelveTestJSON-495634389-project Oct 19 00:45:09.540595 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: method: > Oct 19 00:45:09.540595 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: exception: Instance b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2 could not be found. Oct 19 00:45:09.540595 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: action_args: {'id': 'b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2'} Oct 19 00:45:09.542440 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 19 00:45:09.542440 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 19 00:45:09.542440 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 19 00:45:09.542440 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 19 00:45:09.542440 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 19 00:45:09.542440 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 19 00:45:09.542440 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 19 00:45:09.542440 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 19 00:45:09.542440 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 19 00:45:09.542440 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 19 00:45:09.542440 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2 could not be found. Oct 19 00:45:09.542440 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:45:09.542440 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 19 00:45:09.542440 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:45:09.542440 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 19 00:45:09.542440 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 19 00:45:09.542440 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 19 00:45:09.542440 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 19 00:45:09.542440 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 19 00:45:09.542440 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 19 00:45:09.545095 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 19 00:45:09.545095 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 19 00:45:09.545095 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 19 00:45:09.545095 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 19 00:45:09.545095 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 19 00:45:09.545095 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 19 00:45:09.545095 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 19 00:45:09.545095 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2 could not be found. Oct 19 00:45:09.545095 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:45:09.545095 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 19 00:45:09.545095 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:45:09.545095 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 19 00:45:09.545095 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 19 00:45:09.545095 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 19 00:45:09.545095 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 19 00:45:09.545095 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 19 00:45:09.545095 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2 could not be found. Oct 19 00:45:09.545095 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:45:09.548604 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-56726637-824c-4335-834a-7a8bc41b04a8 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Returning 404 to user: Instance b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2 could not be found. {{(pid=125641) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 19 00:45:09.548604 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-56726637-824c-4335-834a-7a8bc41b04a8 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] 10.222.0.26 "GET /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2" status: 404 len: 111 microversion: 2.20 time: 0.041381 Oct 19 00:45:09.548604 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 453/568] 10.222.0.26 () {60 vars in 1277 bytes} [Wed Oct 19 00:45:09 2022] GET /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2 => generated 111 bytes in 44 msecs (HTTP/1.1 404) 9 headers in 381 bytes (2 switches on core 0) Oct 19 00:45:09.554210 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-fac8ff75-e082-4896-b23f-48fb2193c6c9 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Calling method '.version_select at 0x7f71cb621430>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:45:09.579136 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-fac8ff75-e082-4896-b23f-48fb2193c6c9 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] 10.222.0.26 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-2045940183" status: 204 len: 0 microversion: 2.20 time: 0.030917 Oct 19 00:45:09.579901 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 454/569] 10.222.0.26 () {60 vars in 1262 bytes} [Wed Oct 19 00:45:09 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-2045940183 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 9 headers in 365 bytes (2 switches on core 0) Oct 19 00:45:10.421861 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-c7ecd67d-a2a9-47ca-b288-688e8eb9cfbe tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Action: 'create', calling method: .version_select at 0x7f71cb607ca0>, body: {"keypair": {"name": "tempest-keypair-936267011", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBMJ7DW35cnbLR0AuLZPddrUN1zcOkDnPwQLWvac9Lug+0KbKYswX9HIuK4Fvm85N2GTLn77for+T2QsgDSA0KxhotldmykXpXv1YYCJaDTq9LoUe+0s7Pejk8FTHPYG3Ww=="}} {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 19 00:45:10.454610 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.quota [None req-c7ecd67d-a2a9-47ca-b288-688e8eb9cfbe tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Getting quotas for project 37ffcffcae9b41bbb2d13adc49cb99df. Resources: {'key_pairs'} {{(pid=125641) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 19 00:45:10.463395 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.quota [None req-c7ecd67d-a2a9-47ca-b288-688e8eb9cfbe tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Getting quotas for user 69fef5650fff42e4ac4ed3ca74d4b123 and project 37ffcffcae9b41bbb2d13adc49cb99df. Resources: {'key_pairs'} {{(pid=125641) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 19 00:45:10.502055 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-c7ecd67d-a2a9-47ca-b288-688e8eb9cfbe tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] 10.222.0.26 "POST /compute/v2.1/os-keypairs" status: 201 len: 400 microversion: 2.20 time: 0.087999 Oct 19 00:45:10.502707 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 455/570] 10.222.0.26 () {62 vars in 1200 bytes} [Wed Oct 19 00:45:10 2022] POST /compute/v2.1/os-keypairs => generated 400 bytes in 90 msecs (HTTP/1.1 201) 9 headers in 364 bytes (1 switches on core 0) Oct 19 00:45:12.007622 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-1090522a-1498-46e9-8092-e38f32ee433e tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeShelveTestJSON-server-1895111250", "imageRef": "b387b4f8-f75e-4a68-b49c-a015bf61c46b", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "173cfaec-0c5f-4ee8-a8f2-5a3596dcd7bc"}], "security_groups": [{"name": "tempest-securitygroup--1321913039"}], "key_name": "tempest-keypair-936267011", "user_data": "}} {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 19 00:45:12.135553 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.network.neutron [None req-1090522a-1498-46e9-8092-e38f32ee433e tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] validate_networks() for [('173cfaec-0c5f-4ee8-a8f2-5a3596dcd7bc', None, None, None, None, None)] {{(pid=125641) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 19 00:45:12.571434 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.virt.hardware [None req-1090522a-1498-46e9-8092-e38f32ee433e tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Flavor limits 0:0:0 {{(pid=125641) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 19 00:45:12.571903 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.virt.hardware [None req-1090522a-1498-46e9-8092-e38f32ee433e tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Image limits 0:0:0 {{(pid=125641) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 19 00:45:12.572433 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.virt.hardware [None req-1090522a-1498-46e9-8092-e38f32ee433e tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Flavor pref 0:0:0 {{(pid=125641) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 19 00:45:12.572850 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.virt.hardware [None req-1090522a-1498-46e9-8092-e38f32ee433e tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Image pref 0:0:0 {{(pid=125641) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 19 00:45:12.573270 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.virt.hardware [None req-1090522a-1498-46e9-8092-e38f32ee433e tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=125641) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 19 00:45:12.583018 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-1090522a-1498-46e9-8092-e38f32ee433e tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:45:12.583515 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-1090522a-1498-46e9-8092-e38f32ee433e tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:45:12.583955 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-1090522a-1498-46e9-8092-e38f32ee433e tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:45:12.584700 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-1090522a-1498-46e9-8092-e38f32ee433e tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:45:12.585160 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-1090522a-1498-46e9-8092-e38f32ee433e tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:45:12.585592 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-1090522a-1498-46e9-8092-e38f32ee433e tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:45:12.606623 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.quota [None req-1090522a-1498-46e9-8092-e38f32ee433e tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Getting quotas for project 37ffcffcae9b41bbb2d13adc49cb99df. Resources: {'cores', 'ram', 'instances'} {{(pid=125641) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 19 00:45:12.611284 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.quota [None req-1090522a-1498-46e9-8092-e38f32ee433e tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Getting quotas for user 69fef5650fff42e4ac4ed3ca74d4b123 and project 37ffcffcae9b41bbb2d13adc49cb99df. Resources: {'cores', 'ram', 'instances'} {{(pid=125641) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 19 00:45:12.624759 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-1090522a-1498-46e9-8092-e38f32ee433e tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Going to run 1 instances... {{(pid=125641) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 19 00:45:12.627203 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-1090522a-1498-46e9-8092-e38f32ee433e tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] [instance: 85efeeae-0648-4a33-bc74-c0f0d25880aa] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='b387b4f8-f75e-4a68-b49c-a015bf61c46b',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=125641) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 19 00:45:12.680998 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-1090522a-1498-46e9-8092-e38f32ee433e tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 202 len: 402 microversion: 2.20 time: 0.677481 Oct 19 00:45:12.682251 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 456/571] 10.222.0.26 () {62 vars in 1188 bytes} [Wed Oct 19 00:45:12 2022] POST /compute/v2.1/servers => generated 402 bytes in 680 msecs (HTTP/1.1 202) 10 headers in 453 bytes (2 switches on core 0) Oct 19 00:45:12.699272 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-897f9e74-cbb1-4a34-926b-f82884a79a63 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:45:12.700408 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-897f9e74-cbb1-4a34-926b-f82884a79a63 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] [instance: 85efeeae-0648-4a33-bc74-c0f0d25880aa] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:45:12.784725 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-897f9e74-cbb1-4a34-926b-f82884a79a63 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:45:12.793809 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-897f9e74-cbb1-4a34-926b-f82884a79a63 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:45:12.795882 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-897f9e74-cbb1-4a34-926b-f82884a79a63 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:45:12.797281 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-897f9e74-cbb1-4a34-926b-f82884a79a63 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] 10.222.0.26 "GET /compute/v2.1/servers/85efeeae-0648-4a33-bc74-c0f0d25880aa" status: 200 len: 1281 microversion: 2.20 time: 0.107092 Oct 19 00:45:12.797958 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 457/572] 10.222.0.26 () {60 vars in 1277 bytes} [Wed Oct 19 00:45:12 2022] GET /compute/v2.1/servers/85efeeae-0648-4a33-bc74-c0f0d25880aa => generated 1281 bytes in 109 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 19 00:45:13.825162 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-18ec5d6a-1275-46e2-8ece-33537f0db78a tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:45:13.825875 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-18ec5d6a-1275-46e2-8ece-33537f0db78a tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] [instance: 85efeeae-0648-4a33-bc74-c0f0d25880aa] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:45:13.833183 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-18ec5d6a-1275-46e2-8ece-33537f0db78a tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:45:13.833675 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-18ec5d6a-1275-46e2-8ece-33537f0db78a tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:45:13.834122 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-18ec5d6a-1275-46e2-8ece-33537f0db78a tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:45:13.834857 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-18ec5d6a-1275-46e2-8ece-33537f0db78a tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:45:13.835316 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-18ec5d6a-1275-46e2-8ece-33537f0db78a tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:45:13.835743 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-18ec5d6a-1275-46e2-8ece-33537f0db78a tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:45:13.907112 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-18ec5d6a-1275-46e2-8ece-33537f0db78a tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:45:13.920527 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-18ec5d6a-1275-46e2-8ece-33537f0db78a tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:45:13.923425 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-18ec5d6a-1275-46e2-8ece-33537f0db78a tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:45:13.925497 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-18ec5d6a-1275-46e2-8ece-33537f0db78a tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] 10.222.0.26 "GET /compute/v2.1/servers/85efeeae-0648-4a33-bc74-c0f0d25880aa" status: 200 len: 1333 microversion: 2.20 time: 0.105293 Oct 19 00:45:13.926492 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 458/573] 10.222.0.26 () {60 vars in 1277 bytes} [Wed Oct 19 00:45:13 2022] GET /compute/v2.1/servers/85efeeae-0648-4a33-bc74-c0f0d25880aa => generated 1333 bytes in 107 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 19 00:45:14.968292 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-fd9147e0-f290-4429-9303-df1a9290c1cd tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:45:14.968963 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-fd9147e0-f290-4429-9303-df1a9290c1cd tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] [instance: 85efeeae-0648-4a33-bc74-c0f0d25880aa] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:45:14.976397 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-fd9147e0-f290-4429-9303-df1a9290c1cd tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:45:14.976879 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-fd9147e0-f290-4429-9303-df1a9290c1cd tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:45:14.978152 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-fd9147e0-f290-4429-9303-df1a9290c1cd tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:45:14.979331 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-fd9147e0-f290-4429-9303-df1a9290c1cd tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:45:14.979797 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-fd9147e0-f290-4429-9303-df1a9290c1cd tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:45:14.980226 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-fd9147e0-f290-4429-9303-df1a9290c1cd tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:45:15.071242 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-fd9147e0-f290-4429-9303-df1a9290c1cd tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:45:15.088201 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-fd9147e0-f290-4429-9303-df1a9290c1cd tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:45:15.091252 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-fd9147e0-f290-4429-9303-df1a9290c1cd tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:45:15.093363 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-fd9147e0-f290-4429-9303-df1a9290c1cd tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] 10.222.0.26 "GET /compute/v2.1/servers/85efeeae-0648-4a33-bc74-c0f0d25880aa" status: 200 len: 1339 microversion: 2.20 time: 0.128776 Oct 19 00:45:15.095184 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 459/574] 10.222.0.26 () {60 vars in 1277 bytes} [Wed Oct 19 00:45:14 2022] GET /compute/v2.1/servers/85efeeae-0648-4a33-bc74-c0f0d25880aa => generated 1339 bytes in 132 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 19 00:45:16.127714 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-54127d91-19a4-4f94-80d0-25c531c9dd01 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:45:16.129048 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-54127d91-19a4-4f94-80d0-25c531c9dd01 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] [instance: 85efeeae-0648-4a33-bc74-c0f0d25880aa] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:45:16.138750 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-54127d91-19a4-4f94-80d0-25c531c9dd01 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:45:16.139208 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-54127d91-19a4-4f94-80d0-25c531c9dd01 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:45:16.139642 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-54127d91-19a4-4f94-80d0-25c531c9dd01 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:45:16.140682 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-54127d91-19a4-4f94-80d0-25c531c9dd01 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:45:16.141332 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-54127d91-19a4-4f94-80d0-25c531c9dd01 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:45:16.141766 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-54127d91-19a4-4f94-80d0-25c531c9dd01 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:45:16.309748 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-54127d91-19a4-4f94-80d0-25c531c9dd01 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:45:16.319358 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-54127d91-19a4-4f94-80d0-25c531c9dd01 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:45:16.321713 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-54127d91-19a4-4f94-80d0-25c531c9dd01 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:45:16.323214 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-54127d91-19a4-4f94-80d0-25c531c9dd01 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] 10.222.0.26 "GET /compute/v2.1/servers/85efeeae-0648-4a33-bc74-c0f0d25880aa" status: 200 len: 1407 microversion: 2.20 time: 0.201307 Oct 19 00:45:16.323783 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 460/575] 10.222.0.26 () {60 vars in 1277 bytes} [Wed Oct 19 00:45:16 2022] GET /compute/v2.1/servers/85efeeae-0648-4a33-bc74-c0f0d25880aa => generated 1407 bytes in 203 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 19 00:45:16.548276 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [req-ad0428bc-6b30-4f13-8b20-9c61dddf6c1b req-771de46d-fd2e-464b-8621-28e7f1c8f7f3 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "85efeeae-0648-4a33-bc74-c0f0d25880aa", "tag": "ba933942-134c-497e-8475-ff284eee237e"}]} {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 19 00:45:16.560671 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-ad0428bc-6b30-4f13-8b20-9c61dddf6c1b req-771de46d-fd2e-464b-8621-28e7f1c8f7f3 service nova] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:45:16.561599 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-ad0428bc-6b30-4f13-8b20-9c61dddf6c1b req-771de46d-fd2e-464b-8621-28e7f1c8f7f3 service nova] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:45:16.562182 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-ad0428bc-6b30-4f13-8b20-9c61dddf6c1b req-771de46d-fd2e-464b-8621-28e7f1c8f7f3 service nova] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:45:16.576372 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.compute.server_external_events [req-ad0428bc-6b30-4f13-8b20-9c61dddf6c1b req-771de46d-fd2e-464b-8621-28e7f1c8f7f3 service nova] Creating event network-changed:ba933942-134c-497e-8475-ff284eee237e for instance 85efeeae-0648-4a33-bc74-c0f0d25880aa on hv-cinder-maste Oct 19 00:45:16.582469 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [req-ad0428bc-6b30-4f13-8b20-9c61dddf6c1b req-771de46d-fd2e-464b-8621-28e7f1c8f7f3 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.037921 Oct 19 00:45:16.583232 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 461/576] 10.222.0.26 () {64 vars in 1317 bytes} [Wed Oct 19 00:45:16 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 40 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 19 00:45:17.351504 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-96756f9e-d179-4f11-b2cc-16f4dce36ce3 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:45:17.352910 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-96756f9e-d179-4f11-b2cc-16f4dce36ce3 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] [instance: 85efeeae-0648-4a33-bc74-c0f0d25880aa] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:45:17.359708 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-96756f9e-d179-4f11-b2cc-16f4dce36ce3 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:45:17.360200 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-96756f9e-d179-4f11-b2cc-16f4dce36ce3 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:45:17.360639 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-96756f9e-d179-4f11-b2cc-16f4dce36ce3 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:45:17.361357 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-96756f9e-d179-4f11-b2cc-16f4dce36ce3 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:45:17.361812 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-96756f9e-d179-4f11-b2cc-16f4dce36ce3 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:45:17.362237 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-96756f9e-d179-4f11-b2cc-16f4dce36ce3 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:45:17.521470 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-96756f9e-d179-4f11-b2cc-16f4dce36ce3 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:45:17.531296 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-96756f9e-d179-4f11-b2cc-16f4dce36ce3 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:45:17.532924 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-96756f9e-d179-4f11-b2cc-16f4dce36ce3 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:45:17.534106 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-96756f9e-d179-4f11-b2cc-16f4dce36ce3 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] 10.222.0.26 "GET /compute/v2.1/servers/85efeeae-0648-4a33-bc74-c0f0d25880aa" status: 200 len: 1407 microversion: 2.20 time: 0.187959 Oct 19 00:45:17.534890 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 462/577] 10.222.0.26 () {60 vars in 1277 bytes} [Wed Oct 19 00:45:17 2022] GET /compute/v2.1/servers/85efeeae-0648-4a33-bc74-c0f0d25880aa => generated 1407 bytes in 190 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 19 00:45:18.562753 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-bde1e677-d481-4514-8443-755dbc359328 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:45:18.563440 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-bde1e677-d481-4514-8443-755dbc359328 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] [instance: 85efeeae-0648-4a33-bc74-c0f0d25880aa] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:45:18.571529 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-bde1e677-d481-4514-8443-755dbc359328 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:45:18.572170 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-bde1e677-d481-4514-8443-755dbc359328 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:45:18.572768 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-bde1e677-d481-4514-8443-755dbc359328 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:45:18.573579 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-bde1e677-d481-4514-8443-755dbc359328 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:45:18.574124 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-bde1e677-d481-4514-8443-755dbc359328 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:45:18.574744 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-bde1e677-d481-4514-8443-755dbc359328 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:45:18.719234 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-bde1e677-d481-4514-8443-755dbc359328 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:45:18.728791 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-bde1e677-d481-4514-8443-755dbc359328 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:45:18.730249 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-bde1e677-d481-4514-8443-755dbc359328 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:45:18.731460 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-bde1e677-d481-4514-8443-755dbc359328 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] 10.222.0.26 "GET /compute/v2.1/servers/85efeeae-0648-4a33-bc74-c0f0d25880aa" status: 200 len: 1407 microversion: 2.20 time: 0.172825 Oct 19 00:45:18.732025 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 463/578] 10.222.0.26 () {60 vars in 1277 bytes} [Wed Oct 19 00:45:18 2022] GET /compute/v2.1/servers/85efeeae-0648-4a33-bc74-c0f0d25880aa => generated 1407 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 19 00:45:19.756887 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-36201b07-02c0-4e3b-9c21-54ad758621ba tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:45:19.757653 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-36201b07-02c0-4e3b-9c21-54ad758621ba tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] [instance: 85efeeae-0648-4a33-bc74-c0f0d25880aa] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:45:19.765088 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-36201b07-02c0-4e3b-9c21-54ad758621ba tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:45:19.765575 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-36201b07-02c0-4e3b-9c21-54ad758621ba tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:45:19.766066 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-36201b07-02c0-4e3b-9c21-54ad758621ba tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:45:19.766828 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-36201b07-02c0-4e3b-9c21-54ad758621ba tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:45:19.767282 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-36201b07-02c0-4e3b-9c21-54ad758621ba tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:45:19.767706 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-36201b07-02c0-4e3b-9c21-54ad758621ba tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:45:19.917813 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-36201b07-02c0-4e3b-9c21-54ad758621ba tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:45:19.931634 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-36201b07-02c0-4e3b-9c21-54ad758621ba tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:45:19.933132 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-36201b07-02c0-4e3b-9c21-54ad758621ba tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:45:19.934472 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-36201b07-02c0-4e3b-9c21-54ad758621ba tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] 10.222.0.26 "GET /compute/v2.1/servers/85efeeae-0648-4a33-bc74-c0f0d25880aa" status: 200 len: 1407 microversion: 2.20 time: 0.181598 Oct 19 00:45:19.935432 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 464/579] 10.222.0.26 () {60 vars in 1277 bytes} [Wed Oct 19 00:45:19 2022] GET /compute/v2.1/servers/85efeeae-0648-4a33-bc74-c0f0d25880aa => generated 1407 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 19 00:45:20.959760 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-448a9b88-9d5e-44e4-bf8e-2c455aa4a6f6 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:45:20.960432 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-448a9b88-9d5e-44e4-bf8e-2c455aa4a6f6 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] [instance: 85efeeae-0648-4a33-bc74-c0f0d25880aa] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:45:20.967366 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-448a9b88-9d5e-44e4-bf8e-2c455aa4a6f6 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:45:20.967847 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-448a9b88-9d5e-44e4-bf8e-2c455aa4a6f6 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:45:20.968278 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-448a9b88-9d5e-44e4-bf8e-2c455aa4a6f6 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:45:20.968983 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-448a9b88-9d5e-44e4-bf8e-2c455aa4a6f6 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:45:20.969429 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-448a9b88-9d5e-44e4-bf8e-2c455aa4a6f6 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:45:20.970098 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-448a9b88-9d5e-44e4-bf8e-2c455aa4a6f6 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:45:21.140340 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-448a9b88-9d5e-44e4-bf8e-2c455aa4a6f6 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:45:21.155987 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-448a9b88-9d5e-44e4-bf8e-2c455aa4a6f6 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:45:21.158201 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-448a9b88-9d5e-44e4-bf8e-2c455aa4a6f6 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:45:21.159446 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-448a9b88-9d5e-44e4-bf8e-2c455aa4a6f6 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] 10.222.0.26 "GET /compute/v2.1/servers/85efeeae-0648-4a33-bc74-c0f0d25880aa" status: 200 len: 1407 microversion: 2.20 time: 0.203577 Oct 19 00:45:21.160047 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 465/580] 10.222.0.26 () {60 vars in 1277 bytes} [Wed Oct 19 00:45:20 2022] GET /compute/v2.1/servers/85efeeae-0648-4a33-bc74-c0f0d25880aa => generated 1407 bytes in 205 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 19 00:45:22.190401 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-d570d5ea-b53d-40f0-ad3e-4ff62f1a2533 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:45:22.191564 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-d570d5ea-b53d-40f0-ad3e-4ff62f1a2533 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] [instance: 85efeeae-0648-4a33-bc74-c0f0d25880aa] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:45:22.200182 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d570d5ea-b53d-40f0-ad3e-4ff62f1a2533 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:45:22.200682 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d570d5ea-b53d-40f0-ad3e-4ff62f1a2533 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:45:22.201123 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d570d5ea-b53d-40f0-ad3e-4ff62f1a2533 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:45:22.201836 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d570d5ea-b53d-40f0-ad3e-4ff62f1a2533 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:45:22.202289 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d570d5ea-b53d-40f0-ad3e-4ff62f1a2533 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:45:22.202740 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d570d5ea-b53d-40f0-ad3e-4ff62f1a2533 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:45:22.351603 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-d570d5ea-b53d-40f0-ad3e-4ff62f1a2533 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:45:22.360454 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-d570d5ea-b53d-40f0-ad3e-4ff62f1a2533 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:45:22.361916 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-d570d5ea-b53d-40f0-ad3e-4ff62f1a2533 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:45:22.363111 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-d570d5ea-b53d-40f0-ad3e-4ff62f1a2533 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] 10.222.0.26 "GET /compute/v2.1/servers/85efeeae-0648-4a33-bc74-c0f0d25880aa" status: 200 len: 1407 microversion: 2.20 time: 0.180401 Oct 19 00:45:22.363676 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 466/581] 10.222.0.26 () {60 vars in 1277 bytes} [Wed Oct 19 00:45:22 2022] GET /compute/v2.1/servers/85efeeae-0648-4a33-bc74-c0f0d25880aa => generated 1407 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 19 00:45:23.389742 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-c1b85bc7-ffab-4c79-b2f3-7681081b4ea7 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:45:23.390586 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-c1b85bc7-ffab-4c79-b2f3-7681081b4ea7 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] [instance: 85efeeae-0648-4a33-bc74-c0f0d25880aa] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:45:23.392340 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG dbcounter [-] [125641] Writing DB stats nova_cell0:SELECT=2 {{(pid=125641) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 00:45:23.393070 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG dbcounter [-] [125641] Writing DB stats nova_cell0:SELECT=12 {{(pid=125641) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 00:45:23.394195 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG dbcounter [-] [125641] Writing DB stats nova_api:SELECT=94,nova_api:UPDATE=1,nova_api:DELETE=1,nova_api:INSERT=4 {{(pid=125641) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 00:45:23.402243 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-c1b85bc7-ffab-4c79-b2f3-7681081b4ea7 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:45:23.402783 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-c1b85bc7-ffab-4c79-b2f3-7681081b4ea7 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:45:23.403231 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-c1b85bc7-ffab-4c79-b2f3-7681081b4ea7 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:45:23.403968 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-c1b85bc7-ffab-4c79-b2f3-7681081b4ea7 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:45:23.404426 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-c1b85bc7-ffab-4c79-b2f3-7681081b4ea7 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:45:23.404854 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-c1b85bc7-ffab-4c79-b2f3-7681081b4ea7 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:45:23.563959 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-c1b85bc7-ffab-4c79-b2f3-7681081b4ea7 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:45:23.572944 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-c1b85bc7-ffab-4c79-b2f3-7681081b4ea7 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:45:23.574377 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-c1b85bc7-ffab-4c79-b2f3-7681081b4ea7 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:45:23.575546 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-c1b85bc7-ffab-4c79-b2f3-7681081b4ea7 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] 10.222.0.26 "GET /compute/v2.1/servers/85efeeae-0648-4a33-bc74-c0f0d25880aa" status: 200 len: 1407 microversion: 2.20 time: 0.189912 Oct 19 00:45:23.576103 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 467/582] 10.222.0.26 () {60 vars in 1277 bytes} [Wed Oct 19 00:45:23 2022] GET /compute/v2.1/servers/85efeeae-0648-4a33-bc74-c0f0d25880aa => generated 1407 bytes in 191 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 19 00:45:24.602759 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-27533a8f-652f-42f6-aa7e-474595169ac6 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:45:24.603843 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-27533a8f-652f-42f6-aa7e-474595169ac6 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] [instance: 85efeeae-0648-4a33-bc74-c0f0d25880aa] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:45:24.612960 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-27533a8f-652f-42f6-aa7e-474595169ac6 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:45:24.613801 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-27533a8f-652f-42f6-aa7e-474595169ac6 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:45:24.614360 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-27533a8f-652f-42f6-aa7e-474595169ac6 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:45:24.615204 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-27533a8f-652f-42f6-aa7e-474595169ac6 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:45:24.615732 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-27533a8f-652f-42f6-aa7e-474595169ac6 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:45:24.616225 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-27533a8f-652f-42f6-aa7e-474595169ac6 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:45:24.890241 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-27533a8f-652f-42f6-aa7e-474595169ac6 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:45:24.895813 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG dbcounter [-] [125641] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=125641) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 00:45:24.903785 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-27533a8f-652f-42f6-aa7e-474595169ac6 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:45:24.905265 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-27533a8f-652f-42f6-aa7e-474595169ac6 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:45:24.906420 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-27533a8f-652f-42f6-aa7e-474595169ac6 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] 10.222.0.26 "GET /compute/v2.1/servers/85efeeae-0648-4a33-bc74-c0f0d25880aa" status: 200 len: 1407 microversion: 2.20 time: 0.308081 Oct 19 00:45:24.907024 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 468/583] 10.222.0.26 () {60 vars in 1277 bytes} [Wed Oct 19 00:45:24 2022] GET /compute/v2.1/servers/85efeeae-0648-4a33-bc74-c0f0d25880aa => generated 1407 bytes in 309 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 19 00:45:25.932085 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-5cea3e96-f6c2-4f88-98b7-1f5e98f538ae tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:45:25.932833 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-5cea3e96-f6c2-4f88-98b7-1f5e98f538ae tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] [instance: 85efeeae-0648-4a33-bc74-c0f0d25880aa] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:45:25.943625 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-5cea3e96-f6c2-4f88-98b7-1f5e98f538ae tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:45:25.944188 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-5cea3e96-f6c2-4f88-98b7-1f5e98f538ae tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:45:25.944765 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-5cea3e96-f6c2-4f88-98b7-1f5e98f538ae tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:45:25.946190 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-5cea3e96-f6c2-4f88-98b7-1f5e98f538ae tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:45:25.946772 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-5cea3e96-f6c2-4f88-98b7-1f5e98f538ae tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:45:25.947293 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-5cea3e96-f6c2-4f88-98b7-1f5e98f538ae tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:45:26.091834 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-5cea3e96-f6c2-4f88-98b7-1f5e98f538ae tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:45:26.107858 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-5cea3e96-f6c2-4f88-98b7-1f5e98f538ae tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:45:26.109735 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-5cea3e96-f6c2-4f88-98b7-1f5e98f538ae tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:45:26.110997 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-5cea3e96-f6c2-4f88-98b7-1f5e98f538ae tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] 10.222.0.26 "GET /compute/v2.1/servers/85efeeae-0648-4a33-bc74-c0f0d25880aa" status: 200 len: 1407 microversion: 2.20 time: 0.183509 Oct 19 00:45:26.111573 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 469/584] 10.222.0.26 () {60 vars in 1277 bytes} [Wed Oct 19 00:45:25 2022] GET /compute/v2.1/servers/85efeeae-0648-4a33-bc74-c0f0d25880aa => generated 1407 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 19 00:45:26.523208 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [req-9ab57303-2977-4a31-a588-d062d203bdad req-ffe090cc-2ca4-44c0-9b28-0c4fb76cb2b2 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "85efeeae-0648-4a33-bc74-c0f0d25880aa", "name": "network-vif-plugged", "status": "completed", "tag": "ba933942-134c-497e-8475-ff284eee237e"}]} {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 19 00:45:26.536868 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-9ab57303-2977-4a31-a588-d062d203bdad req-ffe090cc-2ca4-44c0-9b28-0c4fb76cb2b2 service nova] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:45:26.537358 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-9ab57303-2977-4a31-a588-d062d203bdad req-ffe090cc-2ca4-44c0-9b28-0c4fb76cb2b2 service nova] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:45:26.538169 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-9ab57303-2977-4a31-a588-d062d203bdad req-ffe090cc-2ca4-44c0-9b28-0c4fb76cb2b2 service nova] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:45:26.551810 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.compute.server_external_events [req-9ab57303-2977-4a31-a588-d062d203bdad req-ffe090cc-2ca4-44c0-9b28-0c4fb76cb2b2 service nova] Creating event network-vif-plugged:ba933942-134c-497e-8475-ff284eee237e for instance 85efeeae-0648-4a33-bc74-c0f0d25880aa on hv-cinder-maste Oct 19 00:45:26.558662 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [req-9ab57303-2977-4a31-a588-d062d203bdad req-ffe090cc-2ca4-44c0-9b28-0c4fb76cb2b2 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.041725 Oct 19 00:45:26.560023 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 470/585] 10.222.0.26 () {64 vars in 1317 bytes} [Wed Oct 19 00:45:26 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 19 00:45:27.138640 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-089002f7-f149-40c3-a853-f703ce248b67 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:45:27.139382 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-089002f7-f149-40c3-a853-f703ce248b67 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] [instance: 85efeeae-0648-4a33-bc74-c0f0d25880aa] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:45:27.147002 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-089002f7-f149-40c3-a853-f703ce248b67 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:45:27.147575 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-089002f7-f149-40c3-a853-f703ce248b67 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:45:27.148038 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-089002f7-f149-40c3-a853-f703ce248b67 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:45:27.149570 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-089002f7-f149-40c3-a853-f703ce248b67 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:45:27.150206 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-089002f7-f149-40c3-a853-f703ce248b67 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:45:27.150646 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-089002f7-f149-40c3-a853-f703ce248b67 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:45:27.293302 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-089002f7-f149-40c3-a853-f703ce248b67 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:45:27.302683 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-089002f7-f149-40c3-a853-f703ce248b67 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:45:27.304148 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-089002f7-f149-40c3-a853-f703ce248b67 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:45:27.305325 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-089002f7-f149-40c3-a853-f703ce248b67 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] 10.222.0.26 "GET /compute/v2.1/servers/85efeeae-0648-4a33-bc74-c0f0d25880aa" status: 200 len: 1407 microversion: 2.20 time: 0.171724 Oct 19 00:45:27.306049 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 471/586] 10.222.0.26 () {60 vars in 1277 bytes} [Wed Oct 19 00:45:27 2022] GET /compute/v2.1/servers/85efeeae-0648-4a33-bc74-c0f0d25880aa => generated 1407 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 19 00:45:28.330962 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-c66d1363-656b-4142-a2d2-596091745f1c tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:45:28.331684 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-c66d1363-656b-4142-a2d2-596091745f1c tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] [instance: 85efeeae-0648-4a33-bc74-c0f0d25880aa] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:45:28.339221 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-c66d1363-656b-4142-a2d2-596091745f1c tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:45:28.339725 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-c66d1363-656b-4142-a2d2-596091745f1c tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:45:28.340173 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-c66d1363-656b-4142-a2d2-596091745f1c tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:45:28.340906 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-c66d1363-656b-4142-a2d2-596091745f1c tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:45:28.341372 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-c66d1363-656b-4142-a2d2-596091745f1c tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:45:28.341804 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-c66d1363-656b-4142-a2d2-596091745f1c tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:45:28.491738 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-c66d1363-656b-4142-a2d2-596091745f1c tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:45:28.501221 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-c66d1363-656b-4142-a2d2-596091745f1c tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:45:28.502738 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-c66d1363-656b-4142-a2d2-596091745f1c tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:45:28.503856 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-c66d1363-656b-4142-a2d2-596091745f1c tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] 10.222.0.26 "GET /compute/v2.1/servers/85efeeae-0648-4a33-bc74-c0f0d25880aa" status: 200 len: 1407 microversion: 2.20 time: 0.177147 Oct 19 00:45:28.504419 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 472/587] 10.222.0.26 () {60 vars in 1277 bytes} [Wed Oct 19 00:45:28 2022] GET /compute/v2.1/servers/85efeeae-0648-4a33-bc74-c0f0d25880aa => generated 1407 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 19 00:45:29.032564 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-19c7a114-b9d4-492d-9015-6eecc56852e7 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:45:29.033112 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-19c7a114-b9d4-492d-9015-6eecc56852e7 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] [instance: c7dd981d-4382-47dc-8d9c-effae6f630f3] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:45:29.040013 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-19c7a114-b9d4-492d-9015-6eecc56852e7 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:45:29.040661 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-19c7a114-b9d4-492d-9015-6eecc56852e7 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:45:29.041113 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-19c7a114-b9d4-492d-9015-6eecc56852e7 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:45:29.041828 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-19c7a114-b9d4-492d-9015-6eecc56852e7 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:45:29.042286 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-19c7a114-b9d4-492d-9015-6eecc56852e7 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:45:29.042730 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-19c7a114-b9d4-492d-9015-6eecc56852e7 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:45:29.147592 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.objects.instance [None req-19c7a114-b9d4-492d-9015-6eecc56852e7 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lazy-loading 'services' on Instance uuid c7dd981d-4382-47dc-8d9c-effae6f630f3 {{(pid=125641) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 19 00:45:29.249888 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-19c7a114-b9d4-492d-9015-6eecc56852e7 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] 10.222.0.26 "DELETE /compute/v2.1/servers/c7dd981d-4382-47dc-8d9c-effae6f630f3/os-volume_attachments/06efee89-ad39-4197-948d-1ef3be836331" status: 202 len: 0 microversion: 2.1 time: 0.220760 Oct 19 00:45:29.251021 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 473/588] 10.222.0.26 () {58 vars in 1416 bytes} [Wed Oct 19 00:45:29 2022] DELETE /compute/v2.1/servers/c7dd981d-4382-47dc-8d9c-effae6f630f3/os-volume_attachments/06efee89-ad39-4197-948d-1ef3be836331 => generated 0 bytes in 223 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Oct 19 00:45:29.530683 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-5bce3c9b-9ba3-420b-88ce-a06f2cf0a4eb tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:45:29.531497 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-5bce3c9b-9ba3-420b-88ce-a06f2cf0a4eb tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] [instance: 85efeeae-0648-4a33-bc74-c0f0d25880aa] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:45:29.537677 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-5bce3c9b-9ba3-420b-88ce-a06f2cf0a4eb tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:45:29.538156 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-5bce3c9b-9ba3-420b-88ce-a06f2cf0a4eb tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:45:29.538622 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-5bce3c9b-9ba3-420b-88ce-a06f2cf0a4eb tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:45:29.539889 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-5bce3c9b-9ba3-420b-88ce-a06f2cf0a4eb tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:45:29.540343 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-5bce3c9b-9ba3-420b-88ce-a06f2cf0a4eb tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:45:29.540773 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-5bce3c9b-9ba3-420b-88ce-a06f2cf0a4eb tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:45:29.676718 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-5bce3c9b-9ba3-420b-88ce-a06f2cf0a4eb tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:45:29.686198 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-5bce3c9b-9ba3-420b-88ce-a06f2cf0a4eb tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:45:29.687669 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-5bce3c9b-9ba3-420b-88ce-a06f2cf0a4eb tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:45:29.688856 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-5bce3c9b-9ba3-420b-88ce-a06f2cf0a4eb tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] 10.222.0.26 "GET /compute/v2.1/servers/85efeeae-0648-4a33-bc74-c0f0d25880aa" status: 200 len: 1598 microversion: 2.20 time: 0.163130 Oct 19 00:45:29.689432 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 474/589] 10.222.0.26 () {60 vars in 1277 bytes} [Wed Oct 19 00:45:29 2022] GET /compute/v2.1/servers/85efeeae-0648-4a33-bc74-c0f0d25880aa => generated 1598 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 19 00:45:29.714824 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-eb172e3d-7f42-435f-9f63-34a359f2c5fe tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:45:29.715323 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-eb172e3d-7f42-435f-9f63-34a359f2c5fe tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] [instance: 85efeeae-0648-4a33-bc74-c0f0d25880aa] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:45:29.721860 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-eb172e3d-7f42-435f-9f63-34a359f2c5fe tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:45:29.722350 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-eb172e3d-7f42-435f-9f63-34a359f2c5fe tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:45:29.722890 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-eb172e3d-7f42-435f-9f63-34a359f2c5fe tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:45:29.723603 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-eb172e3d-7f42-435f-9f63-34a359f2c5fe tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:45:29.724058 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-eb172e3d-7f42-435f-9f63-34a359f2c5fe tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:45:29.724480 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-eb172e3d-7f42-435f-9f63-34a359f2c5fe tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:45:29.808419 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-eb172e3d-7f42-435f-9f63-34a359f2c5fe tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] 10.222.0.26 "GET /compute/v2.1/servers/85efeeae-0648-4a33-bc74-c0f0d25880aa/os-interface" status: 200 len: 284 microversion: 2.20 time: 0.098200 Oct 19 00:45:29.809191 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 475/590] 10.222.0.26 () {60 vars in 1316 bytes} [Wed Oct 19 00:45:29 2022] GET /compute/v2.1/servers/85efeeae-0648-4a33-bc74-c0f0d25880aa/os-interface => generated 284 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Oct 19 00:45:30.403731 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-abfc9609-52c2-4c1e-a281-1cdc7e22b412 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:45:30.404690 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-abfc9609-52c2-4c1e-a281-1cdc7e22b412 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] [instance: c7dd981d-4382-47dc-8d9c-effae6f630f3] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:45:30.411081 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-abfc9609-52c2-4c1e-a281-1cdc7e22b412 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:45:30.411565 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-abfc9609-52c2-4c1e-a281-1cdc7e22b412 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:45:30.411993 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-abfc9609-52c2-4c1e-a281-1cdc7e22b412 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:45:30.412970 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-abfc9609-52c2-4c1e-a281-1cdc7e22b412 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:45:30.413649 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-abfc9609-52c2-4c1e-a281-1cdc7e22b412 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:45:30.414099 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-abfc9609-52c2-4c1e-a281-1cdc7e22b412 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:45:30.442363 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-abfc9609-52c2-4c1e-a281-1cdc7e22b412 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] [instance: c7dd981d-4382-47dc-8d9c-effae6f630f3] Going to try to terminate instance {{(pid=125641) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 19 00:45:30.527081 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-abfc9609-52c2-4c1e-a281-1cdc7e22b412 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] 10.222.0.26 "DELETE /compute/v2.1/servers/c7dd981d-4382-47dc-8d9c-effae6f630f3" status: 204 len: 0 microversion: 2.1 time: 0.127841 Oct 19 00:45:30.528082 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 476/591] 10.222.0.26 () {58 vars in 1239 bytes} [Wed Oct 19 00:45:30 2022] DELETE /compute/v2.1/servers/c7dd981d-4382-47dc-8d9c-effae6f630f3 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 19 00:45:30.537451 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-736b6d4d-6b2a-4ed6-95f8-60ba3d147f22 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:45:30.539049 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-736b6d4d-6b2a-4ed6-95f8-60ba3d147f22 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] [instance: c7dd981d-4382-47dc-8d9c-effae6f630f3] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:45:30.548825 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-736b6d4d-6b2a-4ed6-95f8-60ba3d147f22 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:45:30.549326 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-736b6d4d-6b2a-4ed6-95f8-60ba3d147f22 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:45:30.549783 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-736b6d4d-6b2a-4ed6-95f8-60ba3d147f22 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:45:30.551085 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-736b6d4d-6b2a-4ed6-95f8-60ba3d147f22 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:45:30.551579 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-736b6d4d-6b2a-4ed6-95f8-60ba3d147f22 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:45:30.552021 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-736b6d4d-6b2a-4ed6-95f8-60ba3d147f22 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:45:30.793454 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-736b6d4d-6b2a-4ed6-95f8-60ba3d147f22 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '39de6a6fff7c4cc08cc62aa86182d71d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1235d8b054b3405595fca761f08b73ae', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:45:30.801670 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-736b6d4d-6b2a-4ed6-95f8-60ba3d147f22 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] 10.222.0.26 "GET /compute/v2.1/servers/c7dd981d-4382-47dc-8d9c-effae6f630f3" status: 200 len: 1513 microversion: 2.1 time: 0.268748 Oct 19 00:45:30.802375 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 477/592] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:45:30 2022] GET /compute/v2.1/servers/c7dd981d-4382-47dc-8d9c-effae6f630f3 => generated 1513 bytes in 270 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:45:31.228100 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [req-48d7e6f3-5e7e-4372-9613-94eaecd1046c req-49237b35-a739-4faa-a468-f5643c2f713d service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "85efeeae-0648-4a33-bc74-c0f0d25880aa", "tag": "ba933942-134c-497e-8475-ff284eee237e"}]} {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 19 00:45:31.241777 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-48d7e6f3-5e7e-4372-9613-94eaecd1046c req-49237b35-a739-4faa-a468-f5643c2f713d service nova] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:45:31.243240 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-48d7e6f3-5e7e-4372-9613-94eaecd1046c req-49237b35-a739-4faa-a468-f5643c2f713d service nova] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:45:31.243638 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-48d7e6f3-5e7e-4372-9613-94eaecd1046c req-49237b35-a739-4faa-a468-f5643c2f713d service nova] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:45:31.256027 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.compute.server_external_events [req-48d7e6f3-5e7e-4372-9613-94eaecd1046c req-49237b35-a739-4faa-a468-f5643c2f713d service nova] Creating event network-changed:ba933942-134c-497e-8475-ff284eee237e for instance 85efeeae-0648-4a33-bc74-c0f0d25880aa on hv-cinder-maste Oct 19 00:45:31.261621 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [req-48d7e6f3-5e7e-4372-9613-94eaecd1046c req-49237b35-a739-4faa-a468-f5643c2f713d service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.038250 Oct 19 00:45:31.262530 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 478/593] 10.222.0.26 () {64 vars in 1317 bytes} [Wed Oct 19 00:45:31 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 40 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 19 00:45:31.832128 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-f0f2aade-84d3-462a-a195-5c627ba957fc tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:45:31.833359 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-f0f2aade-84d3-462a-a195-5c627ba957fc tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] [instance: c7dd981d-4382-47dc-8d9c-effae6f630f3] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:45:31.845379 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-f0f2aade-84d3-462a-a195-5c627ba957fc tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:45:31.846237 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-f0f2aade-84d3-462a-a195-5c627ba957fc tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:45:31.847043 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-f0f2aade-84d3-462a-a195-5c627ba957fc tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:45:31.848297 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-f0f2aade-84d3-462a-a195-5c627ba957fc tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:45:31.849144 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-f0f2aade-84d3-462a-a195-5c627ba957fc tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:45:31.849911 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-f0f2aade-84d3-462a-a195-5c627ba957fc tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:45:32.036048 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-f0f2aade-84d3-462a-a195-5c627ba957fc tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '39de6a6fff7c4cc08cc62aa86182d71d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1235d8b054b3405595fca761f08b73ae', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:45:32.051588 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-f0f2aade-84d3-462a-a195-5c627ba957fc tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] 10.222.0.26 "GET /compute/v2.1/servers/c7dd981d-4382-47dc-8d9c-effae6f630f3" status: 200 len: 1513 microversion: 2.1 time: 0.227992 Oct 19 00:45:32.052593 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 479/594] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:45:31 2022] GET /compute/v2.1/servers/c7dd981d-4382-47dc-8d9c-effae6f630f3 => generated 1513 bytes in 231 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:45:33.081494 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-346d3fef-6bc2-45ad-bdba-0ea765498917 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:45:33.082713 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-346d3fef-6bc2-45ad-bdba-0ea765498917 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] [instance: c7dd981d-4382-47dc-8d9c-effae6f630f3] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:45:33.095391 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-346d3fef-6bc2-45ad-bdba-0ea765498917 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:45:33.096287 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-346d3fef-6bc2-45ad-bdba-0ea765498917 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:45:33.097072 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-346d3fef-6bc2-45ad-bdba-0ea765498917 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:45:33.098360 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-346d3fef-6bc2-45ad-bdba-0ea765498917 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:45:33.099238 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-346d3fef-6bc2-45ad-bdba-0ea765498917 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:45:33.100016 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-346d3fef-6bc2-45ad-bdba-0ea765498917 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:45:33.344470 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-346d3fef-6bc2-45ad-bdba-0ea765498917 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '39de6a6fff7c4cc08cc62aa86182d71d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1235d8b054b3405595fca761f08b73ae', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:45:33.353992 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-346d3fef-6bc2-45ad-bdba-0ea765498917 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] 10.222.0.26 "GET /compute/v2.1/servers/c7dd981d-4382-47dc-8d9c-effae6f630f3" status: 200 len: 1513 microversion: 2.1 time: 0.280099 Oct 19 00:45:33.354710 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 480/595] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:45:33 2022] GET /compute/v2.1/servers/c7dd981d-4382-47dc-8d9c-effae6f630f3 => generated 1513 bytes in 282 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:45:33.571554 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [req-db2ceabe-2d9c-404c-a78b-05d9c49a4e57 req-a582e845-0f89-4d3a-9abd-21b71328ae55 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "c7dd981d-4382-47dc-8d9c-effae6f630f3", "name": "network-vif-deleted", "tag": "7c972af5-e79c-417f-bd49-ea410c70f201"}]} {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 19 00:45:33.589393 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-db2ceabe-2d9c-404c-a78b-05d9c49a4e57 req-a582e845-0f89-4d3a-9abd-21b71328ae55 service nova] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:45:33.589907 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-db2ceabe-2d9c-404c-a78b-05d9c49a4e57 req-a582e845-0f89-4d3a-9abd-21b71328ae55 service nova] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:45:33.590346 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-db2ceabe-2d9c-404c-a78b-05d9c49a4e57 req-a582e845-0f89-4d3a-9abd-21b71328ae55 service nova] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:45:33.603471 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.compute.server_external_events [req-db2ceabe-2d9c-404c-a78b-05d9c49a4e57 req-a582e845-0f89-4d3a-9abd-21b71328ae55 service nova] Creating event network-vif-deleted:7c972af5-e79c-417f-bd49-ea410c70f201 for instance c7dd981d-4382-47dc-8d9c-effae6f630f3 on hv-cinder-maste Oct 19 00:45:33.609657 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [req-db2ceabe-2d9c-404c-a78b-05d9c49a4e57 req-a582e845-0f89-4d3a-9abd-21b71328ae55 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.043728 Oct 19 00:45:33.610404 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 481/596] 10.222.0.26 () {64 vars in 1317 bytes} [Wed Oct 19 00:45:33 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 19 00:45:34.380867 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-dd112b6d-0c6d-455f-bc41-3c8e7a37fa34 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:45:34.381781 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-dd112b6d-0c6d-455f-bc41-3c8e7a37fa34 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] [instance: c7dd981d-4382-47dc-8d9c-effae6f630f3] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:45:34.396965 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-dd112b6d-0c6d-455f-bc41-3c8e7a37fa34 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:45:34.399514 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-dd112b6d-0c6d-455f-bc41-3c8e7a37fa34 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:45:34.399514 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-dd112b6d-0c6d-455f-bc41-3c8e7a37fa34 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:45:34.401559 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-dd112b6d-0c6d-455f-bc41-3c8e7a37fa34 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:45:34.401843 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-dd112b6d-0c6d-455f-bc41-3c8e7a37fa34 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:45:34.402640 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-dd112b6d-0c6d-455f-bc41-3c8e7a37fa34 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:45:34.463777 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-dd112b6d-0c6d-455f-bc41-3c8e7a37fa34 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:45:34.464269 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-dd112b6d-0c6d-455f-bc41-3c8e7a37fa34 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:45:34.464704 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-dd112b6d-0c6d-455f-bc41-3c8e7a37fa34 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:45:34.465162 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.objects.instance [None req-dd112b6d-0c6d-455f-bc41-3c8e7a37fa34 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lazy-loading 'fault' on Instance uuid c7dd981d-4382-47dc-8d9c-effae6f630f3 {{(pid=125641) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 19 00:45:34.541379 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-dd112b6d-0c6d-455f-bc41-3c8e7a37fa34 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '39de6a6fff7c4cc08cc62aa86182d71d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1235d8b054b3405595fca761f08b73ae', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:45:34.554525 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-dd112b6d-0c6d-455f-bc41-3c8e7a37fa34 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] 10.222.0.26 "GET /compute/v2.1/servers/c7dd981d-4382-47dc-8d9c-effae6f630f3" status: 200 len: 1308 microversion: 2.1 time: 0.179132 Oct 19 00:45:34.555933 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 482/597] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:45:34 2022] GET /compute/v2.1/servers/c7dd981d-4382-47dc-8d9c-effae6f630f3 => generated 1308 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:45:35.584471 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-528f15ec-236e-4584-b5d8-813356babb48 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:45:35.585241 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-528f15ec-236e-4584-b5d8-813356babb48 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] [instance: c7dd981d-4382-47dc-8d9c-effae6f630f3] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:45:35.594166 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-528f15ec-236e-4584-b5d8-813356babb48 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:45:35.595062 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-528f15ec-236e-4584-b5d8-813356babb48 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:45:35.595391 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-528f15ec-236e-4584-b5d8-813356babb48 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:45:35.596112 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-528f15ec-236e-4584-b5d8-813356babb48 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:45:35.596576 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-528f15ec-236e-4584-b5d8-813356babb48 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:45:35.597004 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-528f15ec-236e-4584-b5d8-813356babb48 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:45:35.613234 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.wsgi [None req-528f15ec-236e-4584-b5d8-813356babb48 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] HTTP exception thrown: Instance c7dd981d-4382-47dc-8d9c-effae6f630f3 could not be found. Oct 19 00:45:35.615116 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-528f15ec-236e-4584-b5d8-813356babb48 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Request method failure captured: Oct 19 00:45:35.615116 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: request: GET /compute/v2.1/servers/c7dd981d-4382-47dc-8d9c-effae6f630f3 HTTP/1.1 Oct 19 00:45:35.615116 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Accept: application/json Oct 19 00:45:35.615116 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Accept-Encoding: identity Oct 19 00:45:35.615116 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Connection: close Oct 19 00:45:35.615116 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Content-Length: 0 Oct 19 00:45:35.615116 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Content-Type: application/json Oct 19 00:45:35.615116 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Host: 10.222.0.26 Oct 19 00:45:35.615116 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Openstack-System-Scope: None Oct 19 00:45:35.615116 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: User-Agent: python-urllib3/1.26.12 Oct 19 00:45:35.615116 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Auth-Token: gAAAAABjT0gKnENal7-Rh73WviQBQSLL5676mOhmhgH_YQb-aYaRiQLWVmjdDva3tE-fi-9DMsXk14oSgyRxVtRsla_-rGKcqb59timblqX-EY620cGLwQo3rRgl2mWDPAr19L4z-8pjV12JYT9LTCQHYeyjGdQSstwReJL2x7Na4aYUu9YdbF0 Oct 19 00:45:35.615116 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Domain-Id: None Oct 19 00:45:35.615116 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Domain-Name: None Oct 19 00:45:35.615116 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Identity-Status: Confirmed Oct 19 00:45:35.615116 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Is-Admin-Project: True Oct 19 00:45:35.615116 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Project-Domain-Id: default Oct 19 00:45:35.615116 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Project-Domain-Name: Default Oct 19 00:45:35.615116 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Project-Id: 1235d8b054b3405595fca761f08b73ae Oct 19 00:45:35.615116 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Project-Name: tempest-VolumesSnapshotTestJSON-2142930389 Oct 19 00:45:35.615116 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Role: member,reader Oct 19 00:45:35.615116 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Roles: member,reader Oct 19 00:45:35.618337 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/1235d8b054b3405595fca761f08b73ae"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/1235d8b054b3405595fca761f08b73ae"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/1235d8b054b3405595fca761f08b73ae"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_1235d8b054b3405595fca761f08b73ae"}]}] Oct 19 00:45:35.618337 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Tenant: tempest-VolumesSnapshotTestJSON-2142930389 Oct 19 00:45:35.618337 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Tenant-Id: 1235d8b054b3405595fca761f08b73ae Oct 19 00:45:35.618337 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Tenant-Name: tempest-VolumesSnapshotTestJSON-2142930389 Oct 19 00:45:35.618337 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User: tempest-VolumesSnapshotTestJSON-2142930389-project Oct 19 00:45:35.618337 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User-Domain-Id: default Oct 19 00:45:35.618337 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User-Domain-Name: Default Oct 19 00:45:35.618337 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User-Id: 39de6a6fff7c4cc08cc62aa86182d71d Oct 19 00:45:35.618337 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User-Name: tempest-VolumesSnapshotTestJSON-2142930389-project Oct 19 00:45:35.618337 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: method: > Oct 19 00:45:35.618337 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: exception: Instance c7dd981d-4382-47dc-8d9c-effae6f630f3 could not be found. Oct 19 00:45:35.618337 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: action_args: {'id': 'c7dd981d-4382-47dc-8d9c-effae6f630f3'} Oct 19 00:45:35.619375 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 19 00:45:35.619375 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 19 00:45:35.619375 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 19 00:45:35.619375 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 19 00:45:35.619375 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 19 00:45:35.619375 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 19 00:45:35.619375 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 19 00:45:35.619375 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 19 00:45:35.619375 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 19 00:45:35.619375 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 19 00:45:35.619375 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance c7dd981d-4382-47dc-8d9c-effae6f630f3 could not be found. Oct 19 00:45:35.619375 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:45:35.619375 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 19 00:45:35.619375 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:45:35.619375 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 19 00:45:35.619375 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 19 00:45:35.619375 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 19 00:45:35.619375 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 19 00:45:35.619375 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 19 00:45:35.619375 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 19 00:45:35.620747 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 19 00:45:35.620747 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 19 00:45:35.620747 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 19 00:45:35.620747 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 19 00:45:35.620747 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 19 00:45:35.620747 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 19 00:45:35.620747 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 19 00:45:35.620747 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance c7dd981d-4382-47dc-8d9c-effae6f630f3 could not be found. Oct 19 00:45:35.620747 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:45:35.620747 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 19 00:45:35.620747 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:45:35.620747 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 19 00:45:35.620747 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 19 00:45:35.620747 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 19 00:45:35.620747 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 19 00:45:35.620747 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 19 00:45:35.620747 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance c7dd981d-4382-47dc-8d9c-effae6f630f3 could not be found. Oct 19 00:45:35.620747 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:45:35.622414 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-528f15ec-236e-4584-b5d8-813356babb48 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] Returning 404 to user: Instance c7dd981d-4382-47dc-8d9c-effae6f630f3 could not be found. {{(pid=125641) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 19 00:45:35.622414 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-528f15ec-236e-4584-b5d8-813356babb48 tempest-VolumesSnapshotTestJSON-2142930389 tempest-VolumesSnapshotTestJSON-2142930389-project] 10.222.0.26 "GET /compute/v2.1/servers/c7dd981d-4382-47dc-8d9c-effae6f630f3" status: 404 len: 111 microversion: 2.1 time: 0.038756 Oct 19 00:45:35.622414 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 483/598] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:45:35 2022] GET /compute/v2.1/servers/c7dd981d-4382-47dc-8d9c-effae6f630f3 => generated 111 bytes in 41 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Oct 19 00:45:40.231545 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-aa268a19-82e9-4c6b-83a2-0f04d94ca0e2 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:45:40.232294 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-aa268a19-82e9-4c6b-83a2-0f04d94ca0e2 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] [instance: f6051269-1d16-42db-8323-67a532dbcf40] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:45:40.239895 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-aa268a19-82e9-4c6b-83a2-0f04d94ca0e2 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:45:40.240477 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-aa268a19-82e9-4c6b-83a2-0f04d94ca0e2 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:45:40.240992 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-aa268a19-82e9-4c6b-83a2-0f04d94ca0e2 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:45:40.241753 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-aa268a19-82e9-4c6b-83a2-0f04d94ca0e2 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:45:40.242902 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-aa268a19-82e9-4c6b-83a2-0f04d94ca0e2 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:45:40.243462 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-aa268a19-82e9-4c6b-83a2-0f04d94ca0e2 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:45:40.346744 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.objects.instance [None req-aa268a19-82e9-4c6b-83a2-0f04d94ca0e2 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lazy-loading 'services' on Instance uuid f6051269-1d16-42db-8323-67a532dbcf40 {{(pid=125641) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 19 00:45:40.450891 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-aa268a19-82e9-4c6b-83a2-0f04d94ca0e2 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] 10.222.0.26 "DELETE /compute/v2.1/servers/f6051269-1d16-42db-8323-67a532dbcf40/os-volume_attachments/b9abcc63-2fc1-4c36-a503-af404a15c9f5" status: 202 len: 0 microversion: 2.1 time: 0.223337 Oct 19 00:45:40.450891 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 484/599] 10.222.0.26 () {58 vars in 1416 bytes} [Wed Oct 19 00:45:40 2022] DELETE /compute/v2.1/servers/f6051269-1d16-42db-8323-67a532dbcf40/os-volume_attachments/b9abcc63-2fc1-4c36-a503-af404a15c9f5 => generated 0 bytes in 225 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Oct 19 00:45:40.458804 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-dd348f88-9e2b-4f9c-9a14-140b46ed92c3 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:45:40.461299 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-dd348f88-9e2b-4f9c-9a14-140b46ed92c3 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] [instance: f6051269-1d16-42db-8323-67a532dbcf40] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:45:40.468326 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-dd348f88-9e2b-4f9c-9a14-140b46ed92c3 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:45:40.468820 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-dd348f88-9e2b-4f9c-9a14-140b46ed92c3 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:45:40.469255 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-dd348f88-9e2b-4f9c-9a14-140b46ed92c3 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:45:40.470216 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-dd348f88-9e2b-4f9c-9a14-140b46ed92c3 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:45:40.470704 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-dd348f88-9e2b-4f9c-9a14-140b46ed92c3 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:45:40.471158 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-dd348f88-9e2b-4f9c-9a14-140b46ed92c3 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:45:40.477046 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG dbcounter [-] [125641] Writing DB stats nova_cell1:SELECT=96,nova_cell1:INSERT=3,nova_cell1:UPDATE=1 {{(pid=125641) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 00:45:40.505301 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-dd348f88-9e2b-4f9c-9a14-140b46ed92c3 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] 10.222.0.26 "GET /compute/v2.1/servers/f6051269-1d16-42db-8323-67a532dbcf40/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050886 Oct 19 00:45:40.505890 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 485/600] 10.222.0.26 () {58 vars in 1302 bytes} [Wed Oct 19 00:45:40 2022] GET /compute/v2.1/servers/f6051269-1d16-42db-8323-67a532dbcf40/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 19 00:45:41.518760 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-10adedba-9842-48a9-8ec3-bfb0e3842952 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:45:41.521018 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-10adedba-9842-48a9-8ec3-bfb0e3842952 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] [instance: f6051269-1d16-42db-8323-67a532dbcf40] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:45:41.529176 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-10adedba-9842-48a9-8ec3-bfb0e3842952 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:45:41.529683 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-10adedba-9842-48a9-8ec3-bfb0e3842952 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:45:41.530305 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-10adedba-9842-48a9-8ec3-bfb0e3842952 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:45:41.530883 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-10adedba-9842-48a9-8ec3-bfb0e3842952 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:45:41.531348 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-10adedba-9842-48a9-8ec3-bfb0e3842952 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:45:41.531777 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-10adedba-9842-48a9-8ec3-bfb0e3842952 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:45:41.576734 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-10adedba-9842-48a9-8ec3-bfb0e3842952 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] 10.222.0.26 "GET /compute/v2.1/servers/f6051269-1d16-42db-8323-67a532dbcf40/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.061322 Oct 19 00:45:41.577500 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 486/601] 10.222.0.26 () {58 vars in 1302 bytes} [Wed Oct 19 00:45:41 2022] GET /compute/v2.1/servers/f6051269-1d16-42db-8323-67a532dbcf40/os-volume_attachments => generated 197 bytes in 64 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 19 00:45:42.590719 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-52c0072f-8a2d-4f5e-8e3a-3f8be12a9415 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:45:42.592905 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-52c0072f-8a2d-4f5e-8e3a-3f8be12a9415 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] [instance: f6051269-1d16-42db-8323-67a532dbcf40] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:45:42.599906 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-52c0072f-8a2d-4f5e-8e3a-3f8be12a9415 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:45:42.600388 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-52c0072f-8a2d-4f5e-8e3a-3f8be12a9415 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:45:42.600822 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-52c0072f-8a2d-4f5e-8e3a-3f8be12a9415 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:45:42.602512 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-52c0072f-8a2d-4f5e-8e3a-3f8be12a9415 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:45:42.602667 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-52c0072f-8a2d-4f5e-8e3a-3f8be12a9415 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:45:42.603046 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-52c0072f-8a2d-4f5e-8e3a-3f8be12a9415 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:45:42.634599 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-52c0072f-8a2d-4f5e-8e3a-3f8be12a9415 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] 10.222.0.26 "GET /compute/v2.1/servers/f6051269-1d16-42db-8323-67a532dbcf40/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.048961 Oct 19 00:45:42.635188 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 487/602] 10.222.0.26 () {58 vars in 1302 bytes} [Wed Oct 19 00:45:42 2022] GET /compute/v2.1/servers/f6051269-1d16-42db-8323-67a532dbcf40/os-volume_attachments => generated 25 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 356 bytes (2 switches on core 0) Oct 19 00:45:42.776559 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-c0cc4e0f-89c1-45de-bc90-2e0feaba09ab tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:45:42.777065 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-c0cc4e0f-89c1-45de-bc90-2e0feaba09ab tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] [instance: d8ff0e88-597f-4db1-b5f4-852dd33c566d] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:45:42.783292 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-c0cc4e0f-89c1-45de-bc90-2e0feaba09ab tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:45:42.783786 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-c0cc4e0f-89c1-45de-bc90-2e0feaba09ab tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:45:42.784215 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-c0cc4e0f-89c1-45de-bc90-2e0feaba09ab tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:45:42.784926 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-c0cc4e0f-89c1-45de-bc90-2e0feaba09ab tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:45:42.785372 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-c0cc4e0f-89c1-45de-bc90-2e0feaba09ab tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:45:42.785790 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-c0cc4e0f-89c1-45de-bc90-2e0feaba09ab tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:45:42.817771 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-c0cc4e0f-89c1-45de-bc90-2e0feaba09ab tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] 10.222.0.26 "GET /compute/v2.1/servers/d8ff0e88-597f-4db1-b5f4-852dd33c566d/ips" status: 200 len: 149 microversion: 2.1 time: 0.102475 Oct 19 00:45:42.818696 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 488/603] 10.222.0.26 () {58 vars in 1248 bytes} [Wed Oct 19 00:45:42 2022] GET /compute/v2.1/servers/d8ff0e88-597f-4db1-b5f4-852dd33c566d/ips => generated 149 bytes in 105 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 19 00:45:42.825049 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-1d64cee7-f315-45da-bd6c-ea256b912ed1 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:45:42.825557 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-1d64cee7-f315-45da-bd6c-ea256b912ed1 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] [instance: f6051269-1d16-42db-8323-67a532dbcf40] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:45:42.832010 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-1d64cee7-f315-45da-bd6c-ea256b912ed1 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:45:42.832300 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-1d64cee7-f315-45da-bd6c-ea256b912ed1 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:45:42.832714 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-1d64cee7-f315-45da-bd6c-ea256b912ed1 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:45:42.833426 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-1d64cee7-f315-45da-bd6c-ea256b912ed1 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:45:42.833875 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-1d64cee7-f315-45da-bd6c-ea256b912ed1 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:45:42.834296 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-1d64cee7-f315-45da-bd6c-ea256b912ed1 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:45:42.862788 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-1d64cee7-f315-45da-bd6c-ea256b912ed1 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] [instance: f6051269-1d16-42db-8323-67a532dbcf40] Going to try to terminate instance {{(pid=125641) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 19 00:45:42.979888 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-1d64cee7-f315-45da-bd6c-ea256b912ed1 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] 10.222.0.26 "DELETE /compute/v2.1/servers/f6051269-1d16-42db-8323-67a532dbcf40" status: 204 len: 0 microversion: 2.1 time: 0.159504 Oct 19 00:45:42.982631 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 489/604] 10.222.0.26 () {58 vars in 1239 bytes} [Wed Oct 19 00:45:42 2022] DELETE /compute/v2.1/servers/f6051269-1d16-42db-8323-67a532dbcf40 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 19 00:45:42.997084 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-9ea5e4f5-c531-46fa-9deb-0892f1e7ed7a tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:45:42.998637 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-9ea5e4f5-c531-46fa-9deb-0892f1e7ed7a tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] [instance: f6051269-1d16-42db-8323-67a532dbcf40] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:45:43.010462 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-9ea5e4f5-c531-46fa-9deb-0892f1e7ed7a tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:45:43.012374 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-9ea5e4f5-c531-46fa-9deb-0892f1e7ed7a tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:45:43.012766 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-9ea5e4f5-c531-46fa-9deb-0892f1e7ed7a tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:45:43.013694 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-9ea5e4f5-c531-46fa-9deb-0892f1e7ed7a tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:45:43.014173 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-9ea5e4f5-c531-46fa-9deb-0892f1e7ed7a tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:45:43.014642 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-9ea5e4f5-c531-46fa-9deb-0892f1e7ed7a tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:45:43.200855 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-9ea5e4f5-c531-46fa-9deb-0892f1e7ed7a tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd3f0d615187d4779bdd7da8c9f24c796', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0151d58abd047a692f5cc4e7d05bea5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:45:43.221334 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-9ea5e4f5-c531-46fa-9deb-0892f1e7ed7a tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] 10.222.0.26 "GET /compute/v2.1/servers/f6051269-1d16-42db-8323-67a532dbcf40" status: 200 len: 1691 microversion: 2.1 time: 0.232594 Oct 19 00:45:43.222929 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 490/605] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:45:42 2022] GET /compute/v2.1/servers/f6051269-1d16-42db-8323-67a532dbcf40 => generated 1691 bytes in 236 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:45:44.269106 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-9ea00fcc-c7f8-4d0e-aa95-210e0c076b33 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:45:44.270673 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-9ea00fcc-c7f8-4d0e-aa95-210e0c076b33 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] [instance: f6051269-1d16-42db-8323-67a532dbcf40] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:45:44.283707 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-9ea00fcc-c7f8-4d0e-aa95-210e0c076b33 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:45:44.284500 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-9ea00fcc-c7f8-4d0e-aa95-210e0c076b33 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:45:44.285285 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-9ea00fcc-c7f8-4d0e-aa95-210e0c076b33 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:45:44.286619 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-9ea00fcc-c7f8-4d0e-aa95-210e0c076b33 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:45:44.287444 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-9ea00fcc-c7f8-4d0e-aa95-210e0c076b33 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:45:44.288188 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-9ea00fcc-c7f8-4d0e-aa95-210e0c076b33 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:45:44.489211 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-9ea00fcc-c7f8-4d0e-aa95-210e0c076b33 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd3f0d615187d4779bdd7da8c9f24c796', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0151d58abd047a692f5cc4e7d05bea5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:45:44.499390 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-9ea00fcc-c7f8-4d0e-aa95-210e0c076b33 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] 10.222.0.26 "GET /compute/v2.1/servers/f6051269-1d16-42db-8323-67a532dbcf40" status: 200 len: 1691 microversion: 2.1 time: 0.238183 Oct 19 00:45:44.500370 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 491/606] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:45:44 2022] GET /compute/v2.1/servers/f6051269-1d16-42db-8323-67a532dbcf40 => generated 1691 bytes in 240 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:45:44.508548 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-ed5f0372-1b7a-47d5-9255-024b78b08ad1 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "4e5c30c7-fbce-4c05-8cd4-448d0483be3b"}} {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 19 00:45:44.511804 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-ed5f0372-1b7a-47d5-9255-024b78b08ad1 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] [instance: d8ff0e88-597f-4db1-b5f4-852dd33c566d] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:45:44.520304 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ed5f0372-1b7a-47d5-9255-024b78b08ad1 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:45:44.520942 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ed5f0372-1b7a-47d5-9255-024b78b08ad1 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:45:44.521509 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ed5f0372-1b7a-47d5-9255-024b78b08ad1 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:45:44.522324 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ed5f0372-1b7a-47d5-9255-024b78b08ad1 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:45:44.523938 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ed5f0372-1b7a-47d5-9255-024b78b08ad1 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:45:44.524800 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ed5f0372-1b7a-47d5-9255-024b78b08ad1 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:45:49.163399 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-ed5f0372-1b7a-47d5-9255-024b78b08ad1 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] 10.222.0.26 "POST /compute/v2.1/servers/d8ff0e88-597f-4db1-b5f4-852dd33c566d/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 4.661537 Oct 19 00:45:49.164451 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 492/607] 10.222.0.26 () {60 vars in 1323 bytes} [Wed Oct 19 00:45:44 2022] POST /compute/v2.1/servers/d8ff0e88-597f-4db1-b5f4-852dd33c566d/os-volume_attachments => generated 194 bytes in 4663 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 19 00:45:49.170826 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-c33f0b86-2237-4440-997c-dac908231e54 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:45:49.171861 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-c33f0b86-2237-4440-997c-dac908231e54 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] [instance: f6051269-1d16-42db-8323-67a532dbcf40] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:45:49.179960 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-c33f0b86-2237-4440-997c-dac908231e54 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:45:49.180692 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-c33f0b86-2237-4440-997c-dac908231e54 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:45:49.181240 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-c33f0b86-2237-4440-997c-dac908231e54 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:45:49.182121 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-c33f0b86-2237-4440-997c-dac908231e54 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:45:49.182713 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-c33f0b86-2237-4440-997c-dac908231e54 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:45:49.183262 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-c33f0b86-2237-4440-997c-dac908231e54 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:45:49.362403 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-c33f0b86-2237-4440-997c-dac908231e54 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd3f0d615187d4779bdd7da8c9f24c796', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0151d58abd047a692f5cc4e7d05bea5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:45:49.372679 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-c33f0b86-2237-4440-997c-dac908231e54 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] 10.222.0.26 "GET /compute/v2.1/servers/f6051269-1d16-42db-8323-67a532dbcf40" status: 200 len: 1691 microversion: 2.1 time: 0.206794 Oct 19 00:45:49.373454 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 493/608] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:45:49 2022] GET /compute/v2.1/servers/f6051269-1d16-42db-8323-67a532dbcf40 => generated 1691 bytes in 208 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:45:50.403295 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-38b3ef6f-bbe9-4f08-8fa9-b170a737120a tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:45:50.405695 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-38b3ef6f-bbe9-4f08-8fa9-b170a737120a tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] [instance: f6051269-1d16-42db-8323-67a532dbcf40] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:45:50.416161 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-38b3ef6f-bbe9-4f08-8fa9-b170a737120a tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:45:50.416980 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-38b3ef6f-bbe9-4f08-8fa9-b170a737120a tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:45:50.417643 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-38b3ef6f-bbe9-4f08-8fa9-b170a737120a tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:45:50.419308 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-38b3ef6f-bbe9-4f08-8fa9-b170a737120a tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:45:50.420152 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-38b3ef6f-bbe9-4f08-8fa9-b170a737120a tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:45:50.420997 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-38b3ef6f-bbe9-4f08-8fa9-b170a737120a tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:45:50.557563 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-38b3ef6f-bbe9-4f08-8fa9-b170a737120a tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd3f0d615187d4779bdd7da8c9f24c796', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0151d58abd047a692f5cc4e7d05bea5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:45:50.573394 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-38b3ef6f-bbe9-4f08-8fa9-b170a737120a tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] 10.222.0.26 "GET /compute/v2.1/servers/f6051269-1d16-42db-8323-67a532dbcf40" status: 200 len: 1623 microversion: 2.1 time: 0.178782 Oct 19 00:45:50.574233 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 494/609] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:45:50 2022] GET /compute/v2.1/servers/f6051269-1d16-42db-8323-67a532dbcf40 => generated 1623 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:45:50.991851 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [req-6f2b2767-7f71-4453-9221-3b1fa04fd19a req-5f6df961-77f8-4a8d-862e-b133abb0f630 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "f6051269-1d16-42db-8323-67a532dbcf40", "name": "network-vif-deleted", "tag": "667c66be-7422-49b6-8e96-6c36f067b697"}]} {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 19 00:45:51.013813 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-6f2b2767-7f71-4453-9221-3b1fa04fd19a req-5f6df961-77f8-4a8d-862e-b133abb0f630 service nova] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:45:51.015483 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-6f2b2767-7f71-4453-9221-3b1fa04fd19a req-5f6df961-77f8-4a8d-862e-b133abb0f630 service nova] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:45:51.017142 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-6f2b2767-7f71-4453-9221-3b1fa04fd19a req-5f6df961-77f8-4a8d-862e-b133abb0f630 service nova] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:45:51.047081 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.compute.server_external_events [req-6f2b2767-7f71-4453-9221-3b1fa04fd19a req-5f6df961-77f8-4a8d-862e-b133abb0f630 service nova] Creating event network-vif-deleted:667c66be-7422-49b6-8e96-6c36f067b697 for instance f6051269-1d16-42db-8323-67a532dbcf40 on hv-cinder-maste Oct 19 00:45:51.059863 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [req-6f2b2767-7f71-4453-9221-3b1fa04fd19a req-5f6df961-77f8-4a8d-862e-b133abb0f630 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.085325 Oct 19 00:45:51.063061 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 495/610] 10.222.0.26 () {64 vars in 1317 bytes} [Wed Oct 19 00:45:50 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 19 00:45:51.617568 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-cba1d10d-2efd-4607-900f-b304da474566 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:45:51.618621 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-cba1d10d-2efd-4607-900f-b304da474566 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] [instance: f6051269-1d16-42db-8323-67a532dbcf40] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:45:51.626251 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-cba1d10d-2efd-4607-900f-b304da474566 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:45:51.626963 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-cba1d10d-2efd-4607-900f-b304da474566 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:45:51.627658 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-cba1d10d-2efd-4607-900f-b304da474566 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:45:51.628612 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-cba1d10d-2efd-4607-900f-b304da474566 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:45:51.629223 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-cba1d10d-2efd-4607-900f-b304da474566 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:45:51.629768 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-cba1d10d-2efd-4607-900f-b304da474566 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:45:51.673235 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-cba1d10d-2efd-4607-900f-b304da474566 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:45:51.673732 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-cba1d10d-2efd-4607-900f-b304da474566 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:45:51.674208 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-cba1d10d-2efd-4607-900f-b304da474566 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:45:51.674780 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.objects.instance [None req-cba1d10d-2efd-4607-900f-b304da474566 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lazy-loading 'fault' on Instance uuid f6051269-1d16-42db-8323-67a532dbcf40 {{(pid=125641) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 19 00:45:51.744205 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-cba1d10d-2efd-4607-900f-b304da474566 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd3f0d615187d4779bdd7da8c9f24c796', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0151d58abd047a692f5cc4e7d05bea5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:45:51.758856 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-cba1d10d-2efd-4607-900f-b304da474566 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] 10.222.0.26 "GET /compute/v2.1/servers/f6051269-1d16-42db-8323-67a532dbcf40" status: 200 len: 1336 microversion: 2.1 time: 0.146533 Oct 19 00:45:51.771379 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 496/611] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:45:51 2022] GET /compute/v2.1/servers/f6051269-1d16-42db-8323-67a532dbcf40 => generated 1336 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:45:52.801708 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-ba6ddd56-90f8-4f6f-8196-63076c5266e2 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:45:52.802558 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-ba6ddd56-90f8-4f6f-8196-63076c5266e2 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] [instance: f6051269-1d16-42db-8323-67a532dbcf40] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:45:52.814466 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ba6ddd56-90f8-4f6f-8196-63076c5266e2 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:45:52.815002 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ba6ddd56-90f8-4f6f-8196-63076c5266e2 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:45:52.815446 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ba6ddd56-90f8-4f6f-8196-63076c5266e2 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:45:52.816590 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ba6ddd56-90f8-4f6f-8196-63076c5266e2 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:45:52.817322 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ba6ddd56-90f8-4f6f-8196-63076c5266e2 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:45:52.817959 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ba6ddd56-90f8-4f6f-8196-63076c5266e2 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:45:52.833923 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.wsgi [None req-ba6ddd56-90f8-4f6f-8196-63076c5266e2 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] HTTP exception thrown: Instance f6051269-1d16-42db-8323-67a532dbcf40 could not be found. Oct 19 00:45:52.835699 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-ba6ddd56-90f8-4f6f-8196-63076c5266e2 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Request method failure captured: Oct 19 00:45:52.835699 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: request: GET /compute/v2.1/servers/f6051269-1d16-42db-8323-67a532dbcf40 HTTP/1.1 Oct 19 00:45:52.835699 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Accept: application/json Oct 19 00:45:52.835699 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Accept-Encoding: identity Oct 19 00:45:52.835699 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Connection: close Oct 19 00:45:52.835699 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Content-Length: 0 Oct 19 00:45:52.835699 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Content-Type: application/json Oct 19 00:45:52.835699 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Host: 10.222.0.26 Oct 19 00:45:52.835699 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Openstack-System-Scope: None Oct 19 00:45:52.835699 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: User-Agent: python-urllib3/1.26.12 Oct 19 00:45:52.835699 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Auth-Token: gAAAAABjT0eG-h4KFbZ30OiVtvJ2IwzaIfjKNSU1Qsat-O9_Zg1Ouctw-AlAuBWD6kHGuP3EnbU_TJPE1ljWwW1kJo03Mn42ZE5duN2HDm9MXxFr1Oc_Mse4KR4ABtf0tvzP53Pz6nUn07toZIxQKWNLC4yurNeR_6sS4VV90-4qptRCmdstdYU Oct 19 00:45:52.835699 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Domain-Id: None Oct 19 00:45:52.835699 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Domain-Name: None Oct 19 00:45:52.835699 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Identity-Status: Confirmed Oct 19 00:45:52.835699 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Is-Admin-Project: True Oct 19 00:45:52.835699 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Project-Domain-Id: default Oct 19 00:45:52.835699 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Project-Domain-Name: Default Oct 19 00:45:52.835699 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Project-Id: c0151d58abd047a692f5cc4e7d05bea5 Oct 19 00:45:52.835699 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-190719564 Oct 19 00:45:52.835699 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Role: member,reader Oct 19 00:45:52.835699 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Roles: member,reader Oct 19 00:45:52.838133 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/c0151d58abd047a692f5cc4e7d05bea5"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/c0151d58abd047a692f5cc4e7d05bea5"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/c0151d58abd047a692f5cc4e7d05bea5"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_c0151d58abd047a692f5cc4e7d05bea5"}]}] Oct 19 00:45:52.838133 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Tenant: tempest-ServerRescueNegativeTestJSON-190719564 Oct 19 00:45:52.838133 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Tenant-Id: c0151d58abd047a692f5cc4e7d05bea5 Oct 19 00:45:52.838133 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-190719564 Oct 19 00:45:52.838133 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User: tempest-ServerRescueNegativeTestJSON-190719564-project Oct 19 00:45:52.838133 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User-Domain-Id: default Oct 19 00:45:52.838133 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User-Domain-Name: Default Oct 19 00:45:52.838133 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User-Id: d3f0d615187d4779bdd7da8c9f24c796 Oct 19 00:45:52.838133 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User-Name: tempest-ServerRescueNegativeTestJSON-190719564-project Oct 19 00:45:52.838133 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: method: > Oct 19 00:45:52.838133 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: exception: Instance f6051269-1d16-42db-8323-67a532dbcf40 could not be found. Oct 19 00:45:52.838133 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: action_args: {'id': 'f6051269-1d16-42db-8323-67a532dbcf40'} Oct 19 00:45:52.839583 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 19 00:45:52.839583 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 19 00:45:52.839583 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 19 00:45:52.839583 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 19 00:45:52.839583 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 19 00:45:52.839583 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 19 00:45:52.839583 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 19 00:45:52.839583 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 19 00:45:52.839583 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 19 00:45:52.839583 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 19 00:45:52.839583 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance f6051269-1d16-42db-8323-67a532dbcf40 could not be found. Oct 19 00:45:52.839583 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:45:52.839583 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 19 00:45:52.839583 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:45:52.839583 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 19 00:45:52.839583 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 19 00:45:52.839583 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 19 00:45:52.839583 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 19 00:45:52.839583 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 19 00:45:52.839583 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 19 00:45:52.841786 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 19 00:45:52.841786 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 19 00:45:52.841786 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 19 00:45:52.841786 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 19 00:45:52.841786 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 19 00:45:52.841786 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 19 00:45:52.841786 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 19 00:45:52.841786 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance f6051269-1d16-42db-8323-67a532dbcf40 could not be found. Oct 19 00:45:52.841786 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:45:52.841786 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 19 00:45:52.841786 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:45:52.841786 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 19 00:45:52.841786 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 19 00:45:52.841786 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 19 00:45:52.841786 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 19 00:45:52.841786 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 19 00:45:52.841786 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance f6051269-1d16-42db-8323-67a532dbcf40 could not be found. Oct 19 00:45:52.841786 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:45:52.844250 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-ba6ddd56-90f8-4f6f-8196-63076c5266e2 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Returning 404 to user: Instance f6051269-1d16-42db-8323-67a532dbcf40 could not be found. {{(pid=125641) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 19 00:45:52.844250 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-ba6ddd56-90f8-4f6f-8196-63076c5266e2 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] 10.222.0.26 "GET /compute/v2.1/servers/f6051269-1d16-42db-8323-67a532dbcf40" status: 404 len: 111 microversion: 2.1 time: 0.042943 Oct 19 00:45:52.844250 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 497/612] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:45:52 2022] GET /compute/v2.1/servers/f6051269-1d16-42db-8323-67a532dbcf40 => generated 111 bytes in 45 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 19 00:45:52.848354 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-6230a195-8e9e-4032-a040-140e96719338 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Calling method '.version_select at 0x7f71cb5bb4c0>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:45:52.871107 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-6230a195-8e9e-4032-a040-140e96719338 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] 10.222.0.26 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-2094775421" status: 202 len: 0 microversion: 2.1 time: 0.026704 Oct 19 00:45:52.871762 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 498/613] 10.222.0.26 () {58 vars in 1221 bytes} [Wed Oct 19 00:45:52 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-2094775421 => generated 0 bytes in 28 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Oct 19 00:45:56.183593 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-16a00d97-ccae-4986-a10d-46e13093cecd tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:45:56.184725 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-16a00d97-ccae-4986-a10d-46e13093cecd tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] [instance: ab4c73d6-9203-4110-93af-4a78428c672b] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:45:56.187134 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG dbcounter [-] [125641] Writing DB stats nova_api:SELECT=79,nova_api:UPDATE=2,nova_api:DELETE=1 {{(pid=125641) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 00:45:56.200588 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-16a00d97-ccae-4986-a10d-46e13093cecd tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:45:56.201422 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-16a00d97-ccae-4986-a10d-46e13093cecd tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:45:56.202170 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-16a00d97-ccae-4986-a10d-46e13093cecd tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:45:56.203446 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-16a00d97-ccae-4986-a10d-46e13093cecd tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:45:56.204235 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-16a00d97-ccae-4986-a10d-46e13093cecd tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:45:56.204951 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-16a00d97-ccae-4986-a10d-46e13093cecd tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:45:56.239732 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-16a00d97-ccae-4986-a10d-46e13093cecd tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] [instance: ab4c73d6-9203-4110-93af-4a78428c672b] Going to try to terminate instance {{(pid=125641) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 19 00:45:56.298566 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG dbcounter [-] [125641] Writing DB stats nova_cell1:SELECT=94,nova_cell1:UPDATE=4,nova_cell1:INSERT=2 {{(pid=125641) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 00:45:56.353663 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-16a00d97-ccae-4986-a10d-46e13093cecd tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] 10.222.0.26 "DELETE /compute/v2.1/servers/ab4c73d6-9203-4110-93af-4a78428c672b" status: 204 len: 0 microversion: 2.1 time: 0.175664 Oct 19 00:45:56.355222 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 499/614] 10.222.0.26 () {58 vars in 1239 bytes} [Wed Oct 19 00:45:56 2022] DELETE /compute/v2.1/servers/ab4c73d6-9203-4110-93af-4a78428c672b => generated 0 bytes in 178 msecs (HTTP/1.1 204) 9 headers in 363 bytes (2 switches on core 0) Oct 19 00:45:56.369252 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-11b09b51-78b6-45b1-b62b-5ada910fb5e8 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:45:56.370398 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-11b09b51-78b6-45b1-b62b-5ada910fb5e8 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] [instance: ab4c73d6-9203-4110-93af-4a78428c672b] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:45:56.381316 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-11b09b51-78b6-45b1-b62b-5ada910fb5e8 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:45:56.382857 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-11b09b51-78b6-45b1-b62b-5ada910fb5e8 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:45:56.383642 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-11b09b51-78b6-45b1-b62b-5ada910fb5e8 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:45:56.384903 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-11b09b51-78b6-45b1-b62b-5ada910fb5e8 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:45:56.385748 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-11b09b51-78b6-45b1-b62b-5ada910fb5e8 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:45:56.386557 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-11b09b51-78b6-45b1-b62b-5ada910fb5e8 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:45:56.565482 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-11b09b51-78b6-45b1-b62b-5ada910fb5e8 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd3f0d615187d4779bdd7da8c9f24c796', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0151d58abd047a692f5cc4e7d05bea5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:45:56.580173 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-11b09b51-78b6-45b1-b62b-5ada910fb5e8 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] 10.222.0.26 "GET /compute/v2.1/servers/ab4c73d6-9203-4110-93af-4a78428c672b" status: 200 len: 1508 microversion: 2.1 time: 0.219449 Oct 19 00:45:56.581227 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 500/615] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:45:56 2022] GET /compute/v2.1/servers/ab4c73d6-9203-4110-93af-4a78428c672b => generated 1508 bytes in 222 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 19 00:45:56.966874 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-45d02903-9e86-4535-a7bf-1f617651d33f tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "4e5c30c7-fbce-4c05-8cd4-448d0483be3b"}} {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 19 00:45:56.970461 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-45d02903-9e86-4535-a7bf-1f617651d33f tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] [instance: d8ff0e88-597f-4db1-b5f4-852dd33c566d] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:45:56.977957 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-45d02903-9e86-4535-a7bf-1f617651d33f tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:45:56.978465 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-45d02903-9e86-4535-a7bf-1f617651d33f tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:45:56.978936 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-45d02903-9e86-4535-a7bf-1f617651d33f tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:45:56.979698 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-45d02903-9e86-4535-a7bf-1f617651d33f tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:45:56.980155 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-45d02903-9e86-4535-a7bf-1f617651d33f tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:45:56.980634 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-45d02903-9e86-4535-a7bf-1f617651d33f tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:45:57.094969 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.wsgi [None req-45d02903-9e86-4535-a7bf-1f617651d33f tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] HTTP exception thrown: Invalid volume: volume 4e5c30c7-fbce-4c05-8cd4-448d0483be3b is already attached to instances: d8ff0e88-597f-4db1-b5f4-852dd33c566d Oct 19 00:45:57.096738 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-45d02903-9e86-4535-a7bf-1f617651d33f tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Request method failure captured: Oct 19 00:45:57.096738 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: request: POST /compute/v2.1/servers/d8ff0e88-597f-4db1-b5f4-852dd33c566d/os-volume_attachments HTTP/1.1 Oct 19 00:45:57.096738 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Accept: application/json Oct 19 00:45:57.096738 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Accept-Encoding: identity Oct 19 00:45:57.096738 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Connection: close Oct 19 00:45:57.096738 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Content-Length: 74 Oct 19 00:45:57.096738 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Content-Type: application/json Oct 19 00:45:57.096738 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Host: 10.222.0.26 Oct 19 00:45:57.096738 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Openstack-System-Scope: None Oct 19 00:45:57.096738 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: User-Agent: python-urllib3/1.26.12 Oct 19 00:45:57.096738 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Auth-Token: gAAAAABjT0eJXd9ar1txWHw929SSSa4BnzN-f-jxcCDsiqb5Qu6eweguqGUJXghFM8xaTx1ciC3SJFoac87bZHlU5VANYjjzV6ICQChedig2cB1RmNul9h3AYlunCOiAit1Pl73UlpO5HCAxVXGmz0N3G59VsZacZSMYF6RytJzlde2rg7lCpks Oct 19 00:45:57.096738 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Domain-Id: None Oct 19 00:45:57.096738 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Domain-Name: None Oct 19 00:45:57.096738 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Identity-Status: Confirmed Oct 19 00:45:57.096738 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Is-Admin-Project: True Oct 19 00:45:57.096738 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Project-Domain-Id: default Oct 19 00:45:57.096738 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Project-Domain-Name: Default Oct 19 00:45:57.096738 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Project-Id: 0b35c2038f6b4d0a9012b004f1416c00 Oct 19 00:45:57.096738 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Project-Name: tempest-AttachVolumeNegativeTest-1282551484 Oct 19 00:45:57.096738 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Role: member,reader Oct 19 00:45:57.096738 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Roles: member,reader Oct 19 00:45:57.099343 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/0b35c2038f6b4d0a9012b004f1416c00"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/0b35c2038f6b4d0a9012b004f1416c00"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/0b35c2038f6b4d0a9012b004f1416c00"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_0b35c2038f6b4d0a9012b004f1416c00"}]}] Oct 19 00:45:57.099343 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Tenant: tempest-AttachVolumeNegativeTest-1282551484 Oct 19 00:45:57.099343 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Tenant-Id: 0b35c2038f6b4d0a9012b004f1416c00 Oct 19 00:45:57.099343 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1282551484 Oct 19 00:45:57.099343 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User: tempest-AttachVolumeNegativeTest-1282551484-project Oct 19 00:45:57.099343 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User-Domain-Id: default Oct 19 00:45:57.099343 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User-Domain-Name: Default Oct 19 00:45:57.099343 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User-Id: 6f8725c4d05b4dd79b05b43fba15980b Oct 19 00:45:57.099343 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User-Name: tempest-AttachVolumeNegativeTest-1282551484-project Oct 19 00:45:57.099343 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Oct 19 00:45:57.099343 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: {"volumeAttachment": {"volumeId": "4e5c30c7-fbce-4c05-8cd4-448d0483be3b"}} Oct 19 00:45:57.099343 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: method: > Oct 19 00:45:57.100569 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: exception: Invalid volume: volume 4e5c30c7-fbce-4c05-8cd4-448d0483be3b is already attached to instances: d8ff0e88-597f-4db1-b5f4-852dd33c566d Oct 19 00:45:57.100569 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: action_args: {'server_id': 'd8ff0e88-597f-4db1-b5f4-852dd33c566d', 'body': {'volumeAttachment': {'volumeId': '4e5c30c7-fbce-4c05-8cd4-448d0483be3b'}}} Oct 19 00:45:57.100569 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 19 00:45:57.100569 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 19 00:45:57.100569 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 388, in create Oct 19 00:45:57.100569 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi device = self.compute_api.attach_volume( Oct 19 00:45:57.100569 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 241, in inner Oct 19 00:45:57.100569 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Oct 19 00:45:57.100569 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 167, in inner Oct 19 00:45:57.100569 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Oct 19 00:45:57.100569 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5130, in attach_volume Oct 19 00:45:57.100569 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi self._check_volume_already_attached(context, instance, volume) Oct 19 00:45:57.100569 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 4970, in _check_volume_already_attached Oct 19 00:45:57.100569 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=msg) Oct 19 00:45:57.100569 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: volume 4e5c30c7-fbce-4c05-8cd4-448d0483be3b is already attached to instances: d8ff0e88-597f-4db1-b5f4-852dd33c566d Oct 19 00:45:57.100569 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:45:57.100569 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 19 00:45:57.100569 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:45:57.100569 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 19 00:45:57.102223 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 19 00:45:57.102223 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 19 00:45:57.102223 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 19 00:45:57.102223 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 19 00:45:57.102223 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 19 00:45:57.102223 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 19 00:45:57.102223 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 19 00:45:57.102223 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 19 00:45:57.102223 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 19 00:45:57.102223 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 19 00:45:57.102223 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Oct 19 00:45:57.102223 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Oct 19 00:45:57.102223 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 406, in create Oct 19 00:45:57.102223 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Oct 19 00:45:57.102223 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: volume 4e5c30c7-fbce-4c05-8cd4-448d0483be3b is already attached to instances: d8ff0e88-597f-4db1-b5f4-852dd33c566d Oct 19 00:45:57.102223 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:45:57.102223 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 19 00:45:57.102223 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:45:57.102223 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 19 00:45:57.102223 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 19 00:45:57.104635 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 19 00:45:57.104635 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 19 00:45:57.104635 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 19 00:45:57.104635 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: volume 4e5c30c7-fbce-4c05-8cd4-448d0483be3b is already attached to instances: d8ff0e88-597f-4db1-b5f4-852dd33c566d Oct 19 00:45:57.104635 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:45:57.104635 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-45d02903-9e86-4535-a7bf-1f617651d33f tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Returning 400 to user: Invalid volume: volume 4e5c30c7-fbce-4c05-8cd4-448d0483be3b is already attached to instances: d8ff0e88-597f-4db1-b5f4-852dd33c566d {{(pid=125641) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 19 00:45:57.104635 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-45d02903-9e86-4535-a7bf-1f617651d33f tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] 10.222.0.26 "POST /compute/v2.1/servers/d8ff0e88-597f-4db1-b5f4-852dd33c566d/os-volume_attachments" status: 400 len: 174 microversion: 2.1 time: 0.139606 Oct 19 00:45:57.104635 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 501/616] 10.222.0.26 () {60 vars in 1323 bytes} [Wed Oct 19 00:45:56 2022] POST /compute/v2.1/servers/d8ff0e88-597f-4db1-b5f4-852dd33c566d/os-volume_attachments => generated 174 bytes in 141 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Oct 19 00:45:57.181004 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-23becb64-b403-45da-bde7-38e14dd1edf3 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:45:57.181967 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-23becb64-b403-45da-bde7-38e14dd1edf3 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] [instance: d8ff0e88-597f-4db1-b5f4-852dd33c566d] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:45:57.193491 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-23becb64-b403-45da-bde7-38e14dd1edf3 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:45:57.194364 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-23becb64-b403-45da-bde7-38e14dd1edf3 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:45:57.195340 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-23becb64-b403-45da-bde7-38e14dd1edf3 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:45:57.196385 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-23becb64-b403-45da-bde7-38e14dd1edf3 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:45:57.197109 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-23becb64-b403-45da-bde7-38e14dd1edf3 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:45:57.197858 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-23becb64-b403-45da-bde7-38e14dd1edf3 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:45:57.297778 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.objects.instance [None req-23becb64-b403-45da-bde7-38e14dd1edf3 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lazy-loading 'services' on Instance uuid d8ff0e88-597f-4db1-b5f4-852dd33c566d {{(pid=125641) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 19 00:45:57.388573 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-23becb64-b403-45da-bde7-38e14dd1edf3 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] 10.222.0.26 "DELETE /compute/v2.1/servers/d8ff0e88-597f-4db1-b5f4-852dd33c566d/os-volume_attachments/4e5c30c7-fbce-4c05-8cd4-448d0483be3b" status: 202 len: 0 microversion: 2.1 time: 0.215308 Oct 19 00:45:57.389620 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 502/617] 10.222.0.26 () {58 vars in 1416 bytes} [Wed Oct 19 00:45:57 2022] DELETE /compute/v2.1/servers/d8ff0e88-597f-4db1-b5f4-852dd33c566d/os-volume_attachments/4e5c30c7-fbce-4c05-8cd4-448d0483be3b => generated 0 bytes in 218 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Oct 19 00:45:57.400197 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-529af34c-72f1-44cd-9bf0-60b016b9497e tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:45:57.402727 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-529af34c-72f1-44cd-9bf0-60b016b9497e tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] [instance: d8ff0e88-597f-4db1-b5f4-852dd33c566d] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:45:57.409876 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-529af34c-72f1-44cd-9bf0-60b016b9497e tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:45:57.410376 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-529af34c-72f1-44cd-9bf0-60b016b9497e tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:45:57.410849 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-529af34c-72f1-44cd-9bf0-60b016b9497e tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:45:57.411567 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-529af34c-72f1-44cd-9bf0-60b016b9497e tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:45:57.412027 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-529af34c-72f1-44cd-9bf0-60b016b9497e tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:45:57.412468 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-529af34c-72f1-44cd-9bf0-60b016b9497e tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:45:57.446653 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-529af34c-72f1-44cd-9bf0-60b016b9497e tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] 10.222.0.26 "GET /compute/v2.1/servers/d8ff0e88-597f-4db1-b5f4-852dd33c566d/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.052009 Oct 19 00:45:57.447257 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 503/618] 10.222.0.26 () {58 vars in 1302 bytes} [Wed Oct 19 00:45:57 2022] GET /compute/v2.1/servers/d8ff0e88-597f-4db1-b5f4-852dd33c566d/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 19 00:45:57.621104 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-5927934c-651e-4563-92b7-b10118d76620 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:45:57.621718 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-5927934c-651e-4563-92b7-b10118d76620 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] [instance: ab4c73d6-9203-4110-93af-4a78428c672b] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:45:57.628639 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-5927934c-651e-4563-92b7-b10118d76620 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:45:57.629337 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-5927934c-651e-4563-92b7-b10118d76620 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:45:57.629827 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-5927934c-651e-4563-92b7-b10118d76620 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:45:57.630679 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-5927934c-651e-4563-92b7-b10118d76620 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:45:57.631178 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-5927934c-651e-4563-92b7-b10118d76620 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:45:57.631644 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-5927934c-651e-4563-92b7-b10118d76620 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:45:57.793840 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-5927934c-651e-4563-92b7-b10118d76620 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd3f0d615187d4779bdd7da8c9f24c796', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0151d58abd047a692f5cc4e7d05bea5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:45:57.805790 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-5927934c-651e-4563-92b7-b10118d76620 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] 10.222.0.26 "GET /compute/v2.1/servers/ab4c73d6-9203-4110-93af-4a78428c672b" status: 200 len: 1508 microversion: 2.1 time: 0.189150 Oct 19 00:45:57.806395 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 504/619] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:45:57 2022] GET /compute/v2.1/servers/ab4c73d6-9203-4110-93af-4a78428c672b => generated 1508 bytes in 191 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 19 00:45:58.459972 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-057ae0fc-0992-4d10-a534-737c5f903c40 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:45:58.462620 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-057ae0fc-0992-4d10-a534-737c5f903c40 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] [instance: d8ff0e88-597f-4db1-b5f4-852dd33c566d] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:45:58.470142 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-057ae0fc-0992-4d10-a534-737c5f903c40 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:45:58.470681 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-057ae0fc-0992-4d10-a534-737c5f903c40 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:45:58.471122 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-057ae0fc-0992-4d10-a534-737c5f903c40 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:45:58.471857 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-057ae0fc-0992-4d10-a534-737c5f903c40 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:45:58.472318 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-057ae0fc-0992-4d10-a534-737c5f903c40 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:45:58.472911 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-057ae0fc-0992-4d10-a534-737c5f903c40 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:45:58.508564 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-057ae0fc-0992-4d10-a534-737c5f903c40 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] 10.222.0.26 "GET /compute/v2.1/servers/d8ff0e88-597f-4db1-b5f4-852dd33c566d/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.052730 Oct 19 00:45:58.509153 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 505/620] 10.222.0.26 () {58 vars in 1302 bytes} [Wed Oct 19 00:45:58 2022] GET /compute/v2.1/servers/d8ff0e88-597f-4db1-b5f4-852dd33c566d/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 19 00:45:58.837871 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-79eec34c-bc5a-4023-ab29-873d546ad6b3 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:45:58.838426 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-79eec34c-bc5a-4023-ab29-873d546ad6b3 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] [instance: ab4c73d6-9203-4110-93af-4a78428c672b] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:45:58.845132 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-79eec34c-bc5a-4023-ab29-873d546ad6b3 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:45:58.845742 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-79eec34c-bc5a-4023-ab29-873d546ad6b3 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:45:58.846184 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-79eec34c-bc5a-4023-ab29-873d546ad6b3 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:45:58.846926 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-79eec34c-bc5a-4023-ab29-873d546ad6b3 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:45:58.847393 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-79eec34c-bc5a-4023-ab29-873d546ad6b3 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:45:58.847822 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-79eec34c-bc5a-4023-ab29-873d546ad6b3 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:45:59.024425 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-79eec34c-bc5a-4023-ab29-873d546ad6b3 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd3f0d615187d4779bdd7da8c9f24c796', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0151d58abd047a692f5cc4e7d05bea5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:45:59.032686 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-79eec34c-bc5a-4023-ab29-873d546ad6b3 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] 10.222.0.26 "GET /compute/v2.1/servers/ab4c73d6-9203-4110-93af-4a78428c672b" status: 200 len: 1508 microversion: 2.1 time: 0.198691 Oct 19 00:45:59.033265 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 506/621] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:45:58 2022] GET /compute/v2.1/servers/ab4c73d6-9203-4110-93af-4a78428c672b => generated 1508 bytes in 200 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 19 00:45:59.524086 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-d2974c87-b284-4b1a-9c2d-12433c598028 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:45:59.527159 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-d2974c87-b284-4b1a-9c2d-12433c598028 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] [instance: d8ff0e88-597f-4db1-b5f4-852dd33c566d] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:45:59.536729 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d2974c87-b284-4b1a-9c2d-12433c598028 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:45:59.537622 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d2974c87-b284-4b1a-9c2d-12433c598028 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:45:59.538230 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d2974c87-b284-4b1a-9c2d-12433c598028 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:45:59.539159 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d2974c87-b284-4b1a-9c2d-12433c598028 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:45:59.539956 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d2974c87-b284-4b1a-9c2d-12433c598028 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:45:59.540679 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d2974c87-b284-4b1a-9c2d-12433c598028 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:45:59.592688 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-d2974c87-b284-4b1a-9c2d-12433c598028 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] 10.222.0.26 "GET /compute/v2.1/servers/d8ff0e88-597f-4db1-b5f4-852dd33c566d/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.074266 Oct 19 00:45:59.599645 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 507/622] 10.222.0.26 () {58 vars in 1302 bytes} [Wed Oct 19 00:45:59 2022] GET /compute/v2.1/servers/d8ff0e88-597f-4db1-b5f4-852dd33c566d/os-volume_attachments => generated 197 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 19 00:45:59.932832 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [req-c906a37a-38c4-4baa-8986-9d730f890018 req-8fa95da8-2e3c-42c6-8c30-fa72e24a5593 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "ab4c73d6-9203-4110-93af-4a78428c672b", "name": "network-vif-deleted", "tag": "cc5abfca-70ab-406b-974c-c89318ad195e"}]} {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 19 00:45:59.947018 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-c906a37a-38c4-4baa-8986-9d730f890018 req-8fa95da8-2e3c-42c6-8c30-fa72e24a5593 service nova] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:45:59.947580 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-c906a37a-38c4-4baa-8986-9d730f890018 req-8fa95da8-2e3c-42c6-8c30-fa72e24a5593 service nova] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:45:59.948016 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-c906a37a-38c4-4baa-8986-9d730f890018 req-8fa95da8-2e3c-42c6-8c30-fa72e24a5593 service nova] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:45:59.961427 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.compute.server_external_events [req-c906a37a-38c4-4baa-8986-9d730f890018 req-8fa95da8-2e3c-42c6-8c30-fa72e24a5593 service nova] Creating event network-vif-deleted:cc5abfca-70ab-406b-974c-c89318ad195e for instance ab4c73d6-9203-4110-93af-4a78428c672b on hv-cinder-maste Oct 19 00:45:59.966982 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [req-c906a37a-38c4-4baa-8986-9d730f890018 req-8fa95da8-2e3c-42c6-8c30-fa72e24a5593 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.038543 Oct 19 00:45:59.967904 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 508/623] 10.222.0.26 () {64 vars in 1317 bytes} [Wed Oct 19 00:45:59 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 40 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 19 00:46:00.067768 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-b5279ec8-476f-42df-b535-a30fbd6ec862 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:46:00.070467 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-b5279ec8-476f-42df-b535-a30fbd6ec862 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] [instance: ab4c73d6-9203-4110-93af-4a78428c672b] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:46:00.084093 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-b5279ec8-476f-42df-b535-a30fbd6ec862 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:46:00.084792 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-b5279ec8-476f-42df-b535-a30fbd6ec862 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:46:00.085350 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-b5279ec8-476f-42df-b535-a30fbd6ec862 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:46:00.086587 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-b5279ec8-476f-42df-b535-a30fbd6ec862 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:46:00.087146 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-b5279ec8-476f-42df-b535-a30fbd6ec862 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:46:00.087798 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-b5279ec8-476f-42df-b535-a30fbd6ec862 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:46:00.125743 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-b5279ec8-476f-42df-b535-a30fbd6ec862 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:46:00.126252 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-b5279ec8-476f-42df-b535-a30fbd6ec862 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:46:00.126774 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-b5279ec8-476f-42df-b535-a30fbd6ec862 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:46:00.127239 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.objects.instance [None req-b5279ec8-476f-42df-b535-a30fbd6ec862 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lazy-loading 'fault' on Instance uuid ab4c73d6-9203-4110-93af-4a78428c672b {{(pid=125641) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 19 00:46:00.199362 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-b5279ec8-476f-42df-b535-a30fbd6ec862 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd3f0d615187d4779bdd7da8c9f24c796', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0151d58abd047a692f5cc4e7d05bea5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:46:00.213084 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-b5279ec8-476f-42df-b535-a30fbd6ec862 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] 10.222.0.26 "GET /compute/v2.1/servers/ab4c73d6-9203-4110-93af-4a78428c672b" status: 200 len: 1312 microversion: 2.1 time: 0.157227 Oct 19 00:46:00.214103 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 509/624] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:46:00 2022] GET /compute/v2.1/servers/ab4c73d6-9203-4110-93af-4a78428c672b => generated 1312 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:46:00.609563 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-86fc5868-30ee-412e-bc01-ddf9ee64d57d tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:46:00.614292 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-86fc5868-30ee-412e-bc01-ddf9ee64d57d tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] [instance: d8ff0e88-597f-4db1-b5f4-852dd33c566d] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:46:00.624366 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-86fc5868-30ee-412e-bc01-ddf9ee64d57d tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:46:00.624366 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-86fc5868-30ee-412e-bc01-ddf9ee64d57d tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:46:00.624591 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-86fc5868-30ee-412e-bc01-ddf9ee64d57d tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:46:00.625143 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-86fc5868-30ee-412e-bc01-ddf9ee64d57d tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:46:00.625602 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-86fc5868-30ee-412e-bc01-ddf9ee64d57d tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:46:00.626028 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-86fc5868-30ee-412e-bc01-ddf9ee64d57d tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:46:00.682598 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-86fc5868-30ee-412e-bc01-ddf9ee64d57d tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] 10.222.0.26 "GET /compute/v2.1/servers/d8ff0e88-597f-4db1-b5f4-852dd33c566d/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.078080 Oct 19 00:46:00.683826 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 510/625] 10.222.0.26 () {58 vars in 1302 bytes} [Wed Oct 19 00:46:00 2022] GET /compute/v2.1/servers/d8ff0e88-597f-4db1-b5f4-852dd33c566d/os-volume_attachments => generated 25 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 356 bytes (2 switches on core 0) Oct 19 00:46:00.787310 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-76a955f5-4884-4b5a-8d02-b3d351adfb9b tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:46:00.789331 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-76a955f5-4884-4b5a-8d02-b3d351adfb9b tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] [instance: d8ff0e88-597f-4db1-b5f4-852dd33c566d] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:46:00.800435 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-76a955f5-4884-4b5a-8d02-b3d351adfb9b tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:46:00.801323 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-76a955f5-4884-4b5a-8d02-b3d351adfb9b tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:46:00.802402 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-76a955f5-4884-4b5a-8d02-b3d351adfb9b tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:46:00.804677 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-76a955f5-4884-4b5a-8d02-b3d351adfb9b tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:46:00.805403 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-76a955f5-4884-4b5a-8d02-b3d351adfb9b tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:46:00.806381 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-76a955f5-4884-4b5a-8d02-b3d351adfb9b tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:46:00.856752 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-76a955f5-4884-4b5a-8d02-b3d351adfb9b tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] [instance: d8ff0e88-597f-4db1-b5f4-852dd33c566d] Going to try to terminate instance {{(pid=125641) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 19 00:46:01.000517 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-76a955f5-4884-4b5a-8d02-b3d351adfb9b tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] 10.222.0.26 "DELETE /compute/v2.1/servers/d8ff0e88-597f-4db1-b5f4-852dd33c566d" status: 204 len: 0 microversion: 2.1 time: 0.219952 Oct 19 00:46:01.004452 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 511/626] 10.222.0.26 () {58 vars in 1239 bytes} [Wed Oct 19 00:46:00 2022] DELETE /compute/v2.1/servers/d8ff0e88-597f-4db1-b5f4-852dd33c566d => generated 0 bytes in 226 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 19 00:46:01.020317 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-59d11539-945b-4e5c-a9fd-12546e4d098a tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:46:01.021580 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-59d11539-945b-4e5c-a9fd-12546e4d098a tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] [instance: d8ff0e88-597f-4db1-b5f4-852dd33c566d] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:46:01.039164 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-59d11539-945b-4e5c-a9fd-12546e4d098a tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:46:01.040316 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-59d11539-945b-4e5c-a9fd-12546e4d098a tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:46:01.040799 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-59d11539-945b-4e5c-a9fd-12546e4d098a tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:46:01.042710 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-59d11539-945b-4e5c-a9fd-12546e4d098a tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:46:01.043189 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-59d11539-945b-4e5c-a9fd-12546e4d098a tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:46:01.043919 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-59d11539-945b-4e5c-a9fd-12546e4d098a tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:46:01.243120 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-59d11539-945b-4e5c-a9fd-12546e4d098a tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f8725c4d05b4dd79b05b43fba15980b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0b35c2038f6b4d0a9012b004f1416c00', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:46:01.252237 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-59d11539-945b-4e5c-a9fd-12546e4d098a tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] 10.222.0.26 "GET /compute/v2.1/servers/d8ff0e88-597f-4db1-b5f4-852dd33c566d" status: 200 len: 1681 microversion: 2.1 time: 0.239038 Oct 19 00:46:01.252909 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 512/627] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:46:01 2022] GET /compute/v2.1/servers/d8ff0e88-597f-4db1-b5f4-852dd33c566d => generated 1681 bytes in 243 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 19 00:46:01.259489 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-cd5d4ea6-5194-4629-a156-0752ec214da2 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:46:01.260425 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-cd5d4ea6-5194-4629-a156-0752ec214da2 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] [instance: ab4c73d6-9203-4110-93af-4a78428c672b] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:46:01.272487 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-cd5d4ea6-5194-4629-a156-0752ec214da2 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:46:01.273686 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-cd5d4ea6-5194-4629-a156-0752ec214da2 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:46:01.274140 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-cd5d4ea6-5194-4629-a156-0752ec214da2 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:46:01.274904 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-cd5d4ea6-5194-4629-a156-0752ec214da2 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:46:01.275559 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-cd5d4ea6-5194-4629-a156-0752ec214da2 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:46:01.275994 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-cd5d4ea6-5194-4629-a156-0752ec214da2 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:46:01.293685 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.wsgi [None req-cd5d4ea6-5194-4629-a156-0752ec214da2 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] HTTP exception thrown: Instance ab4c73d6-9203-4110-93af-4a78428c672b could not be found. Oct 19 00:46:01.295398 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-cd5d4ea6-5194-4629-a156-0752ec214da2 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Request method failure captured: Oct 19 00:46:01.295398 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: request: GET /compute/v2.1/servers/ab4c73d6-9203-4110-93af-4a78428c672b HTTP/1.1 Oct 19 00:46:01.295398 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Accept: application/json Oct 19 00:46:01.295398 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Accept-Encoding: identity Oct 19 00:46:01.295398 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Connection: close Oct 19 00:46:01.295398 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Content-Length: 0 Oct 19 00:46:01.295398 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Content-Type: application/json Oct 19 00:46:01.295398 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Host: 10.222.0.26 Oct 19 00:46:01.295398 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Openstack-System-Scope: None Oct 19 00:46:01.295398 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: User-Agent: python-urllib3/1.26.12 Oct 19 00:46:01.295398 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Auth-Token: gAAAAABjT0eG-h4KFbZ30OiVtvJ2IwzaIfjKNSU1Qsat-O9_Zg1Ouctw-AlAuBWD6kHGuP3EnbU_TJPE1ljWwW1kJo03Mn42ZE5duN2HDm9MXxFr1Oc_Mse4KR4ABtf0tvzP53Pz6nUn07toZIxQKWNLC4yurNeR_6sS4VV90-4qptRCmdstdYU Oct 19 00:46:01.295398 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Domain-Id: None Oct 19 00:46:01.295398 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Domain-Name: None Oct 19 00:46:01.295398 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Identity-Status: Confirmed Oct 19 00:46:01.295398 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Is-Admin-Project: True Oct 19 00:46:01.295398 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Project-Domain-Id: default Oct 19 00:46:01.295398 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Project-Domain-Name: Default Oct 19 00:46:01.295398 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Project-Id: c0151d58abd047a692f5cc4e7d05bea5 Oct 19 00:46:01.295398 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-190719564 Oct 19 00:46:01.295398 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Role: member,reader Oct 19 00:46:01.295398 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Roles: member,reader Oct 19 00:46:01.298314 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/c0151d58abd047a692f5cc4e7d05bea5"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/c0151d58abd047a692f5cc4e7d05bea5"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/c0151d58abd047a692f5cc4e7d05bea5"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_c0151d58abd047a692f5cc4e7d05bea5"}]}] Oct 19 00:46:01.298314 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Tenant: tempest-ServerRescueNegativeTestJSON-190719564 Oct 19 00:46:01.298314 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Tenant-Id: c0151d58abd047a692f5cc4e7d05bea5 Oct 19 00:46:01.298314 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-190719564 Oct 19 00:46:01.298314 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User: tempest-ServerRescueNegativeTestJSON-190719564-project Oct 19 00:46:01.298314 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User-Domain-Id: default Oct 19 00:46:01.298314 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User-Domain-Name: Default Oct 19 00:46:01.298314 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User-Id: d3f0d615187d4779bdd7da8c9f24c796 Oct 19 00:46:01.298314 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User-Name: tempest-ServerRescueNegativeTestJSON-190719564-project Oct 19 00:46:01.298314 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: method: > Oct 19 00:46:01.298314 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: exception: Instance ab4c73d6-9203-4110-93af-4a78428c672b could not be found. Oct 19 00:46:01.298314 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: action_args: {'id': 'ab4c73d6-9203-4110-93af-4a78428c672b'} Oct 19 00:46:01.300186 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 19 00:46:01.300186 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 19 00:46:01.300186 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 19 00:46:01.300186 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 19 00:46:01.300186 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 19 00:46:01.300186 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 19 00:46:01.300186 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 19 00:46:01.300186 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 19 00:46:01.300186 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 19 00:46:01.300186 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 19 00:46:01.300186 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance ab4c73d6-9203-4110-93af-4a78428c672b could not be found. Oct 19 00:46:01.300186 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:46:01.300186 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 19 00:46:01.300186 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:46:01.300186 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 19 00:46:01.300186 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 19 00:46:01.300186 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 19 00:46:01.300186 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 19 00:46:01.300186 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 19 00:46:01.300186 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 19 00:46:01.302996 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 19 00:46:01.302996 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 19 00:46:01.302996 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 19 00:46:01.302996 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 19 00:46:01.302996 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 19 00:46:01.302996 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 19 00:46:01.302996 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 19 00:46:01.302996 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance ab4c73d6-9203-4110-93af-4a78428c672b could not be found. Oct 19 00:46:01.302996 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:46:01.302996 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 19 00:46:01.302996 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:46:01.302996 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 19 00:46:01.302996 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 19 00:46:01.302996 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 19 00:46:01.302996 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 19 00:46:01.302996 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 19 00:46:01.302996 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance ab4c73d6-9203-4110-93af-4a78428c672b could not be found. Oct 19 00:46:01.302996 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:46:01.305811 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-cd5d4ea6-5194-4629-a156-0752ec214da2 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Returning 404 to user: Instance ab4c73d6-9203-4110-93af-4a78428c672b could not be found. {{(pid=125641) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 19 00:46:01.305811 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-cd5d4ea6-5194-4629-a156-0752ec214da2 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] 10.222.0.26 "GET /compute/v2.1/servers/ab4c73d6-9203-4110-93af-4a78428c672b" status: 404 len: 111 microversion: 2.1 time: 0.042753 Oct 19 00:46:01.305811 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 513/628] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:46:01 2022] GET /compute/v2.1/servers/ab4c73d6-9203-4110-93af-4a78428c672b => generated 111 bytes in 44 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 19 00:46:01.308462 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-bba1f44d-08ed-4eb6-a852-bd03b23b724a tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:46:01.308982 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-bba1f44d-08ed-4eb6-a852-bd03b23b724a tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] [instance: 3da61834-fd67-49d2-925c-c18fc264042c] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:46:01.315582 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-bba1f44d-08ed-4eb6-a852-bd03b23b724a tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:46:01.316276 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-bba1f44d-08ed-4eb6-a852-bd03b23b724a tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:46:01.316726 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-bba1f44d-08ed-4eb6-a852-bd03b23b724a tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:46:01.317446 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-bba1f44d-08ed-4eb6-a852-bd03b23b724a tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:46:01.318432 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-bba1f44d-08ed-4eb6-a852-bd03b23b724a tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:46:01.318896 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-bba1f44d-08ed-4eb6-a852-bd03b23b724a tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:46:01.348646 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-bba1f44d-08ed-4eb6-a852-bd03b23b724a tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] [instance: 3da61834-fd67-49d2-925c-c18fc264042c] Going to try to terminate instance {{(pid=125641) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 19 00:46:01.352157 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG dbcounter [-] [125641] Writing DB stats nova_cell1:SELECT=96,nova_cell1:INSERT=3,nova_cell1:UPDATE=1 {{(pid=125641) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 00:46:01.453516 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-bba1f44d-08ed-4eb6-a852-bd03b23b724a tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] 10.222.0.26 "DELETE /compute/v2.1/servers/3da61834-fd67-49d2-925c-c18fc264042c" status: 204 len: 0 microversion: 2.1 time: 0.149181 Oct 19 00:46:01.454436 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 514/629] 10.222.0.26 () {58 vars in 1239 bytes} [Wed Oct 19 00:46:01 2022] DELETE /compute/v2.1/servers/3da61834-fd67-49d2-925c-c18fc264042c => generated 0 bytes in 151 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 19 00:46:01.465141 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-3a735da6-445c-46ea-8e7c-3ef4011fe6b8 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:46:01.465931 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-3a735da6-445c-46ea-8e7c-3ef4011fe6b8 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] [instance: 3da61834-fd67-49d2-925c-c18fc264042c] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:46:01.474300 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-3a735da6-445c-46ea-8e7c-3ef4011fe6b8 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:46:01.474977 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-3a735da6-445c-46ea-8e7c-3ef4011fe6b8 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:46:01.475568 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-3a735da6-445c-46ea-8e7c-3ef4011fe6b8 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:46:01.476449 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-3a735da6-445c-46ea-8e7c-3ef4011fe6b8 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:46:01.477032 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-3a735da6-445c-46ea-8e7c-3ef4011fe6b8 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:46:01.477585 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-3a735da6-445c-46ea-8e7c-3ef4011fe6b8 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:46:01.690802 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-3a735da6-445c-46ea-8e7c-3ef4011fe6b8 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd3f0d615187d4779bdd7da8c9f24c796', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0151d58abd047a692f5cc4e7d05bea5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:46:01.717499 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-3a735da6-445c-46ea-8e7c-3ef4011fe6b8 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] 10.222.0.26 "GET /compute/v2.1/servers/3da61834-fd67-49d2-925c-c18fc264042c" status: 200 len: 1521 microversion: 2.1 time: 0.257411 Oct 19 00:46:01.718584 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 515/630] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:46:01 2022] GET /compute/v2.1/servers/3da61834-fd67-49d2-925c-c18fc264042c => generated 1521 bytes in 260 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 19 00:46:02.293751 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-513eed69-efb8-495b-a1e5-5a7544c945e7 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:46:02.294517 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-513eed69-efb8-495b-a1e5-5a7544c945e7 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] [instance: d8ff0e88-597f-4db1-b5f4-852dd33c566d] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:46:02.301811 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-513eed69-efb8-495b-a1e5-5a7544c945e7 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:46:02.302307 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-513eed69-efb8-495b-a1e5-5a7544c945e7 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:46:02.302783 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-513eed69-efb8-495b-a1e5-5a7544c945e7 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:46:02.303523 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-513eed69-efb8-495b-a1e5-5a7544c945e7 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:46:02.304079 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-513eed69-efb8-495b-a1e5-5a7544c945e7 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:46:02.304512 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-513eed69-efb8-495b-a1e5-5a7544c945e7 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:46:02.467619 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-513eed69-efb8-495b-a1e5-5a7544c945e7 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f8725c4d05b4dd79b05b43fba15980b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0b35c2038f6b4d0a9012b004f1416c00', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:46:02.478171 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-513eed69-efb8-495b-a1e5-5a7544c945e7 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] 10.222.0.26 "GET /compute/v2.1/servers/d8ff0e88-597f-4db1-b5f4-852dd33c566d" status: 200 len: 1681 microversion: 2.1 time: 0.189382 Oct 19 00:46:02.479062 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 516/631] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:46:02 2022] GET /compute/v2.1/servers/d8ff0e88-597f-4db1-b5f4-852dd33c566d => generated 1681 bytes in 191 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 19 00:46:02.743999 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-b037903d-4a48-499d-b306-daba129c860c tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:46:02.744788 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-b037903d-4a48-499d-b306-daba129c860c tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] [instance: 3da61834-fd67-49d2-925c-c18fc264042c] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:46:02.751997 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-b037903d-4a48-499d-b306-daba129c860c tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:46:02.752493 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-b037903d-4a48-499d-b306-daba129c860c tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:46:02.752935 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-b037903d-4a48-499d-b306-daba129c860c tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:46:02.754192 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-b037903d-4a48-499d-b306-daba129c860c tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:46:02.754887 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-b037903d-4a48-499d-b306-daba129c860c tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:46:02.755453 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-b037903d-4a48-499d-b306-daba129c860c tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:46:02.919877 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-b037903d-4a48-499d-b306-daba129c860c tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd3f0d615187d4779bdd7da8c9f24c796', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0151d58abd047a692f5cc4e7d05bea5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:46:02.928135 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-b037903d-4a48-499d-b306-daba129c860c tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] 10.222.0.26 "GET /compute/v2.1/servers/3da61834-fd67-49d2-925c-c18fc264042c" status: 200 len: 1521 microversion: 2.1 time: 0.189415 Oct 19 00:46:02.929013 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 517/632] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:46:02 2022] GET /compute/v2.1/servers/3da61834-fd67-49d2-925c-c18fc264042c => generated 1521 bytes in 191 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 19 00:46:03.505412 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-9c236ff6-ddd3-4fdb-a132-d1e5826e585f tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:46:03.506078 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-9c236ff6-ddd3-4fdb-a132-d1e5826e585f tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] [instance: d8ff0e88-597f-4db1-b5f4-852dd33c566d] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:46:03.513929 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-9c236ff6-ddd3-4fdb-a132-d1e5826e585f tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:46:03.514469 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-9c236ff6-ddd3-4fdb-a132-d1e5826e585f tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:46:03.514992 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-9c236ff6-ddd3-4fdb-a132-d1e5826e585f tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:46:03.515746 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-9c236ff6-ddd3-4fdb-a132-d1e5826e585f tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:46:03.516240 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-9c236ff6-ddd3-4fdb-a132-d1e5826e585f tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:46:03.516740 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-9c236ff6-ddd3-4fdb-a132-d1e5826e585f tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:46:03.692291 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-9c236ff6-ddd3-4fdb-a132-d1e5826e585f tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f8725c4d05b4dd79b05b43fba15980b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0b35c2038f6b4d0a9012b004f1416c00', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:46:03.701962 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-9c236ff6-ddd3-4fdb-a132-d1e5826e585f tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] 10.222.0.26 "GET /compute/v2.1/servers/d8ff0e88-597f-4db1-b5f4-852dd33c566d" status: 200 len: 1681 microversion: 2.1 time: 0.200367 Oct 19 00:46:03.702815 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 518/633] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:46:03 2022] GET /compute/v2.1/servers/d8ff0e88-597f-4db1-b5f4-852dd33c566d => generated 1681 bytes in 202 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 19 00:46:03.968069 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-f86a1b73-f769-413b-b9e8-3de78786650f tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:46:03.968823 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-f86a1b73-f769-413b-b9e8-3de78786650f tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] [instance: 3da61834-fd67-49d2-925c-c18fc264042c] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:46:03.976842 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-f86a1b73-f769-413b-b9e8-3de78786650f tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:46:03.977340 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-f86a1b73-f769-413b-b9e8-3de78786650f tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:46:03.977776 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-f86a1b73-f769-413b-b9e8-3de78786650f tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:46:03.978512 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-f86a1b73-f769-413b-b9e8-3de78786650f tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:46:03.978975 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-f86a1b73-f769-413b-b9e8-3de78786650f tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:46:03.979397 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-f86a1b73-f769-413b-b9e8-3de78786650f tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:46:04.136805 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-f86a1b73-f769-413b-b9e8-3de78786650f tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd3f0d615187d4779bdd7da8c9f24c796', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0151d58abd047a692f5cc4e7d05bea5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:46:04.146891 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-f86a1b73-f769-413b-b9e8-3de78786650f tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] 10.222.0.26 "GET /compute/v2.1/servers/3da61834-fd67-49d2-925c-c18fc264042c" status: 200 len: 1521 microversion: 2.1 time: 0.182807 Oct 19 00:46:04.147908 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 519/634] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:46:03 2022] GET /compute/v2.1/servers/3da61834-fd67-49d2-925c-c18fc264042c => generated 1521 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 19 00:46:04.727978 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-73457088-72d4-4c23-8276-35e84d52cc22 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:46:04.728677 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-73457088-72d4-4c23-8276-35e84d52cc22 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] [instance: d8ff0e88-597f-4db1-b5f4-852dd33c566d] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:46:04.735455 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-73457088-72d4-4c23-8276-35e84d52cc22 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:46:04.735936 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-73457088-72d4-4c23-8276-35e84d52cc22 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:46:04.736373 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-73457088-72d4-4c23-8276-35e84d52cc22 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:46:04.737079 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-73457088-72d4-4c23-8276-35e84d52cc22 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:46:04.737534 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-73457088-72d4-4c23-8276-35e84d52cc22 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:46:04.737955 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-73457088-72d4-4c23-8276-35e84d52cc22 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:46:04.894639 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-73457088-72d4-4c23-8276-35e84d52cc22 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f8725c4d05b4dd79b05b43fba15980b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0b35c2038f6b4d0a9012b004f1416c00', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:46:04.903433 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-73457088-72d4-4c23-8276-35e84d52cc22 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] 10.222.0.26 "GET /compute/v2.1/servers/d8ff0e88-597f-4db1-b5f4-852dd33c566d" status: 200 len: 1681 microversion: 2.1 time: 0.180025 Oct 19 00:46:04.904113 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 520/635] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:46:04 2022] GET /compute/v2.1/servers/d8ff0e88-597f-4db1-b5f4-852dd33c566d => generated 1681 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 19 00:46:05.172915 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-ad3aed4c-0e57-4fbb-aeff-0c70cce8b9a5 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:46:05.173588 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-ad3aed4c-0e57-4fbb-aeff-0c70cce8b9a5 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] [instance: 3da61834-fd67-49d2-925c-c18fc264042c] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:46:05.182078 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ad3aed4c-0e57-4fbb-aeff-0c70cce8b9a5 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:46:05.182740 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ad3aed4c-0e57-4fbb-aeff-0c70cce8b9a5 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:46:05.183250 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ad3aed4c-0e57-4fbb-aeff-0c70cce8b9a5 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:46:05.184231 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ad3aed4c-0e57-4fbb-aeff-0c70cce8b9a5 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:46:05.184771 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ad3aed4c-0e57-4fbb-aeff-0c70cce8b9a5 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:46:05.185345 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ad3aed4c-0e57-4fbb-aeff-0c70cce8b9a5 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:46:05.353794 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-ad3aed4c-0e57-4fbb-aeff-0c70cce8b9a5 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd3f0d615187d4779bdd7da8c9f24c796', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0151d58abd047a692f5cc4e7d05bea5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:46:05.362305 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-ad3aed4c-0e57-4fbb-aeff-0c70cce8b9a5 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] 10.222.0.26 "GET /compute/v2.1/servers/3da61834-fd67-49d2-925c-c18fc264042c" status: 200 len: 1521 microversion: 2.1 time: 0.193443 Oct 19 00:46:05.362974 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 521/636] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:46:05 2022] GET /compute/v2.1/servers/3da61834-fd67-49d2-925c-c18fc264042c => generated 1521 bytes in 195 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 19 00:46:05.929440 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-cbed580c-56f4-495b-b3f8-90801b13c97b tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:46:05.930285 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-cbed580c-56f4-495b-b3f8-90801b13c97b tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] [instance: d8ff0e88-597f-4db1-b5f4-852dd33c566d] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:46:05.936728 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-cbed580c-56f4-495b-b3f8-90801b13c97b tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:46:05.937274 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-cbed580c-56f4-495b-b3f8-90801b13c97b tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:46:05.937752 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-cbed580c-56f4-495b-b3f8-90801b13c97b tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:46:05.938761 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-cbed580c-56f4-495b-b3f8-90801b13c97b tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:46:05.939425 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-cbed580c-56f4-495b-b3f8-90801b13c97b tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:46:05.939978 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-cbed580c-56f4-495b-b3f8-90801b13c97b tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:46:06.140713 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-cbed580c-56f4-495b-b3f8-90801b13c97b tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f8725c4d05b4dd79b05b43fba15980b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0b35c2038f6b4d0a9012b004f1416c00', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:46:06.150862 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-cbed580c-56f4-495b-b3f8-90801b13c97b tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] 10.222.0.26 "GET /compute/v2.1/servers/d8ff0e88-597f-4db1-b5f4-852dd33c566d" status: 200 len: 1681 microversion: 2.1 time: 0.225596 Oct 19 00:46:06.151465 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 522/637] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:46:05 2022] GET /compute/v2.1/servers/d8ff0e88-597f-4db1-b5f4-852dd33c566d => generated 1681 bytes in 227 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 19 00:46:06.388788 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-2dacd069-d724-49b7-baf9-2313b08e6df2 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:46:06.389536 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-2dacd069-d724-49b7-baf9-2313b08e6df2 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] [instance: 3da61834-fd67-49d2-925c-c18fc264042c] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:46:06.396601 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-2dacd069-d724-49b7-baf9-2313b08e6df2 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:46:06.397089 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-2dacd069-d724-49b7-baf9-2313b08e6df2 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:46:06.398333 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-2dacd069-d724-49b7-baf9-2313b08e6df2 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:46:06.398949 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-2dacd069-d724-49b7-baf9-2313b08e6df2 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:46:06.399360 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-2dacd069-d724-49b7-baf9-2313b08e6df2 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:46:06.399781 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-2dacd069-d724-49b7-baf9-2313b08e6df2 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:46:06.555706 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-2dacd069-d724-49b7-baf9-2313b08e6df2 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd3f0d615187d4779bdd7da8c9f24c796', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0151d58abd047a692f5cc4e7d05bea5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:46:06.565846 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-2dacd069-d724-49b7-baf9-2313b08e6df2 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] 10.222.0.26 "GET /compute/v2.1/servers/3da61834-fd67-49d2-925c-c18fc264042c" status: 200 len: 1521 microversion: 2.1 time: 0.181795 Oct 19 00:46:06.566456 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 523/638] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:46:06 2022] GET /compute/v2.1/servers/3da61834-fd67-49d2-925c-c18fc264042c => generated 1521 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 19 00:46:07.176443 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-3efa5389-51b2-4976-a007-db704f1d0e1a tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:46:07.177752 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-3efa5389-51b2-4976-a007-db704f1d0e1a tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] [instance: d8ff0e88-597f-4db1-b5f4-852dd33c566d] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:46:07.185683 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-3efa5389-51b2-4976-a007-db704f1d0e1a tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:46:07.186173 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-3efa5389-51b2-4976-a007-db704f1d0e1a tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:46:07.186727 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-3efa5389-51b2-4976-a007-db704f1d0e1a tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:46:07.187452 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-3efa5389-51b2-4976-a007-db704f1d0e1a tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:46:07.187906 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-3efa5389-51b2-4976-a007-db704f1d0e1a tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:46:07.188327 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-3efa5389-51b2-4976-a007-db704f1d0e1a tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:46:07.363612 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-3efa5389-51b2-4976-a007-db704f1d0e1a tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f8725c4d05b4dd79b05b43fba15980b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0b35c2038f6b4d0a9012b004f1416c00', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:46:07.383352 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-3efa5389-51b2-4976-a007-db704f1d0e1a tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] 10.222.0.26 "GET /compute/v2.1/servers/d8ff0e88-597f-4db1-b5f4-852dd33c566d" status: 200 len: 1681 microversion: 2.1 time: 0.210963 Oct 19 00:46:07.384871 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 524/639] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:46:07 2022] GET /compute/v2.1/servers/d8ff0e88-597f-4db1-b5f4-852dd33c566d => generated 1681 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 19 00:46:07.903935 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [req-1b8245d5-a283-48a9-88d8-b2cfe81ed597 req-907d7b67-57f9-43a6-a17a-b7d17473d32d service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "3da61834-fd67-49d2-925c-c18fc264042c", "name": "network-vif-deleted", "tag": "da1ae702-fcf6-4388-8f27-6c4ba5eb5fcf"}]} {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 19 00:46:07.924391 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-1b8245d5-a283-48a9-88d8-b2cfe81ed597 req-907d7b67-57f9-43a6-a17a-b7d17473d32d service nova] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:46:07.925611 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-1b8245d5-a283-48a9-88d8-b2cfe81ed597 req-907d7b67-57f9-43a6-a17a-b7d17473d32d service nova] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:46:07.926684 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-1b8245d5-a283-48a9-88d8-b2cfe81ed597 req-907d7b67-57f9-43a6-a17a-b7d17473d32d service nova] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:46:07.949033 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.compute.server_external_events [req-1b8245d5-a283-48a9-88d8-b2cfe81ed597 req-907d7b67-57f9-43a6-a17a-b7d17473d32d service nova] Creating event network-vif-deleted:da1ae702-fcf6-4388-8f27-6c4ba5eb5fcf for instance 3da61834-fd67-49d2-925c-c18fc264042c on hv-cinder-maste Oct 19 00:46:07.958741 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [req-1b8245d5-a283-48a9-88d8-b2cfe81ed597 req-907d7b67-57f9-43a6-a17a-b7d17473d32d service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.464686 Oct 19 00:46:07.959964 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 525/640] 10.222.0.26 () {64 vars in 1317 bytes} [Wed Oct 19 00:46:07 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 468 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 19 00:46:07.971471 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-c4bcb687-bc04-44e2-8e35-35d531678991 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:46:07.973017 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-c4bcb687-bc04-44e2-8e35-35d531678991 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] [instance: 3da61834-fd67-49d2-925c-c18fc264042c] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:46:08.004146 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-c4bcb687-bc04-44e2-8e35-35d531678991 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:46:08.005022 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-c4bcb687-bc04-44e2-8e35-35d531678991 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:46:08.005872 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-c4bcb687-bc04-44e2-8e35-35d531678991 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:46:08.007872 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-c4bcb687-bc04-44e2-8e35-35d531678991 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:46:08.008739 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-c4bcb687-bc04-44e2-8e35-35d531678991 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:46:08.009234 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-c4bcb687-bc04-44e2-8e35-35d531678991 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:46:08.087383 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-c4bcb687-bc04-44e2-8e35-35d531678991 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:46:08.087999 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-c4bcb687-bc04-44e2-8e35-35d531678991 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:46:08.088519 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-c4bcb687-bc04-44e2-8e35-35d531678991 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:46:08.089059 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.objects.instance [None req-c4bcb687-bc04-44e2-8e35-35d531678991 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lazy-loading 'fault' on Instance uuid 3da61834-fd67-49d2-925c-c18fc264042c {{(pid=125641) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 19 00:46:08.152202 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-c4bcb687-bc04-44e2-8e35-35d531678991 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd3f0d615187d4779bdd7da8c9f24c796', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0151d58abd047a692f5cc4e7d05bea5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:46:08.162404 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-c4bcb687-bc04-44e2-8e35-35d531678991 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] 10.222.0.26 "GET /compute/v2.1/servers/3da61834-fd67-49d2-925c-c18fc264042c" status: 200 len: 1310 microversion: 2.1 time: 0.199392 Oct 19 00:46:08.164106 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 526/641] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:46:07 2022] GET /compute/v2.1/servers/3da61834-fd67-49d2-925c-c18fc264042c => generated 1310 bytes in 204 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:46:08.428883 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-52a18f66-6ed2-44c6-b2c7-2fe50a67f1bd tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:46:08.430520 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-52a18f66-6ed2-44c6-b2c7-2fe50a67f1bd tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] [instance: d8ff0e88-597f-4db1-b5f4-852dd33c566d] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:46:08.439062 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-52a18f66-6ed2-44c6-b2c7-2fe50a67f1bd tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:46:08.439870 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-52a18f66-6ed2-44c6-b2c7-2fe50a67f1bd tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:46:08.440316 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-52a18f66-6ed2-44c6-b2c7-2fe50a67f1bd tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:46:08.441061 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-52a18f66-6ed2-44c6-b2c7-2fe50a67f1bd tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:46:08.441515 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-52a18f66-6ed2-44c6-b2c7-2fe50a67f1bd tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:46:08.442142 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-52a18f66-6ed2-44c6-b2c7-2fe50a67f1bd tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:46:08.503408 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-52a18f66-6ed2-44c6-b2c7-2fe50a67f1bd tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:46:08.503912 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-52a18f66-6ed2-44c6-b2c7-2fe50a67f1bd tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:46:08.504461 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-52a18f66-6ed2-44c6-b2c7-2fe50a67f1bd tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:46:08.505060 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.objects.instance [None req-52a18f66-6ed2-44c6-b2c7-2fe50a67f1bd tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lazy-loading 'fault' on Instance uuid d8ff0e88-597f-4db1-b5f4-852dd33c566d {{(pid=125641) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 19 00:46:08.585514 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-52a18f66-6ed2-44c6-b2c7-2fe50a67f1bd tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f8725c4d05b4dd79b05b43fba15980b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0b35c2038f6b4d0a9012b004f1416c00', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:46:08.595481 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-52a18f66-6ed2-44c6-b2c7-2fe50a67f1bd tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] 10.222.0.26 "GET /compute/v2.1/servers/d8ff0e88-597f-4db1-b5f4-852dd33c566d" status: 200 len: 1332 microversion: 2.1 time: 0.173398 Oct 19 00:46:08.606790 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 527/642] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:46:08 2022] GET /compute/v2.1/servers/d8ff0e88-597f-4db1-b5f4-852dd33c566d => generated 1332 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 19 00:46:09.189689 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-82cb473c-cdb6-4eb0-a4c9-60f833c4f6a4 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:46:09.190461 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-82cb473c-cdb6-4eb0-a4c9-60f833c4f6a4 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] [instance: 3da61834-fd67-49d2-925c-c18fc264042c] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:46:09.202819 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-82cb473c-cdb6-4eb0-a4c9-60f833c4f6a4 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:46:09.203801 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-82cb473c-cdb6-4eb0-a4c9-60f833c4f6a4 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:46:09.204289 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-82cb473c-cdb6-4eb0-a4c9-60f833c4f6a4 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:46:09.205168 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-82cb473c-cdb6-4eb0-a4c9-60f833c4f6a4 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:46:09.205704 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-82cb473c-cdb6-4eb0-a4c9-60f833c4f6a4 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:46:09.206213 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-82cb473c-cdb6-4eb0-a4c9-60f833c4f6a4 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:46:09.223912 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.wsgi [None req-82cb473c-cdb6-4eb0-a4c9-60f833c4f6a4 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] HTTP exception thrown: Instance 3da61834-fd67-49d2-925c-c18fc264042c could not be found. Oct 19 00:46:09.226311 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-82cb473c-cdb6-4eb0-a4c9-60f833c4f6a4 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Request method failure captured: Oct 19 00:46:09.226311 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: request: GET /compute/v2.1/servers/3da61834-fd67-49d2-925c-c18fc264042c HTTP/1.1 Oct 19 00:46:09.226311 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Accept: application/json Oct 19 00:46:09.226311 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Accept-Encoding: identity Oct 19 00:46:09.226311 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Connection: close Oct 19 00:46:09.226311 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Content-Length: 0 Oct 19 00:46:09.226311 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Content-Type: application/json Oct 19 00:46:09.226311 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Host: 10.222.0.26 Oct 19 00:46:09.226311 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Openstack-System-Scope: None Oct 19 00:46:09.226311 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: User-Agent: python-urllib3/1.26.12 Oct 19 00:46:09.226311 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Auth-Token: gAAAAABjT0eG-h4KFbZ30OiVtvJ2IwzaIfjKNSU1Qsat-O9_Zg1Ouctw-AlAuBWD6kHGuP3EnbU_TJPE1ljWwW1kJo03Mn42ZE5duN2HDm9MXxFr1Oc_Mse4KR4ABtf0tvzP53Pz6nUn07toZIxQKWNLC4yurNeR_6sS4VV90-4qptRCmdstdYU Oct 19 00:46:09.226311 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Domain-Id: None Oct 19 00:46:09.226311 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Domain-Name: None Oct 19 00:46:09.226311 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Identity-Status: Confirmed Oct 19 00:46:09.226311 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Is-Admin-Project: True Oct 19 00:46:09.226311 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Project-Domain-Id: default Oct 19 00:46:09.226311 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Project-Domain-Name: Default Oct 19 00:46:09.226311 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Project-Id: c0151d58abd047a692f5cc4e7d05bea5 Oct 19 00:46:09.226311 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-190719564 Oct 19 00:46:09.226311 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Role: member,reader Oct 19 00:46:09.226311 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Roles: member,reader Oct 19 00:46:09.227936 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/c0151d58abd047a692f5cc4e7d05bea5"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/c0151d58abd047a692f5cc4e7d05bea5"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/c0151d58abd047a692f5cc4e7d05bea5"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_c0151d58abd047a692f5cc4e7d05bea5"}]}] Oct 19 00:46:09.227936 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Tenant: tempest-ServerRescueNegativeTestJSON-190719564 Oct 19 00:46:09.227936 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Tenant-Id: c0151d58abd047a692f5cc4e7d05bea5 Oct 19 00:46:09.227936 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-190719564 Oct 19 00:46:09.227936 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User: tempest-ServerRescueNegativeTestJSON-190719564-project Oct 19 00:46:09.227936 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User-Domain-Id: default Oct 19 00:46:09.227936 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User-Domain-Name: Default Oct 19 00:46:09.227936 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User-Id: d3f0d615187d4779bdd7da8c9f24c796 Oct 19 00:46:09.227936 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User-Name: tempest-ServerRescueNegativeTestJSON-190719564-project Oct 19 00:46:09.227936 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: method: > Oct 19 00:46:09.227936 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: exception: Instance 3da61834-fd67-49d2-925c-c18fc264042c could not be found. Oct 19 00:46:09.227936 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: action_args: {'id': '3da61834-fd67-49d2-925c-c18fc264042c'} Oct 19 00:46:09.228820 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 19 00:46:09.228820 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 19 00:46:09.228820 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 19 00:46:09.228820 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 19 00:46:09.228820 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 19 00:46:09.228820 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 19 00:46:09.228820 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 19 00:46:09.228820 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 19 00:46:09.228820 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 19 00:46:09.228820 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 19 00:46:09.228820 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 3da61834-fd67-49d2-925c-c18fc264042c could not be found. Oct 19 00:46:09.228820 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:46:09.228820 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 19 00:46:09.228820 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:46:09.228820 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 19 00:46:09.228820 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 19 00:46:09.228820 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 19 00:46:09.228820 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 19 00:46:09.228820 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 19 00:46:09.228820 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 19 00:46:09.230292 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 19 00:46:09.230292 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 19 00:46:09.230292 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 19 00:46:09.230292 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 19 00:46:09.230292 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 19 00:46:09.230292 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 19 00:46:09.230292 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 19 00:46:09.230292 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 3da61834-fd67-49d2-925c-c18fc264042c could not be found. Oct 19 00:46:09.230292 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:46:09.230292 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 19 00:46:09.230292 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:46:09.230292 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 19 00:46:09.230292 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 19 00:46:09.230292 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 19 00:46:09.230292 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 19 00:46:09.230292 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 19 00:46:09.230292 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 3da61834-fd67-49d2-925c-c18fc264042c could not be found. Oct 19 00:46:09.230292 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:46:09.232730 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-82cb473c-cdb6-4eb0-a4c9-60f833c4f6a4 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] Returning 404 to user: Instance 3da61834-fd67-49d2-925c-c18fc264042c could not be found. {{(pid=125641) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 19 00:46:09.233625 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-82cb473c-cdb6-4eb0-a4c9-60f833c4f6a4 tempest-ServerRescueNegativeTestJSON-190719564 tempest-ServerRescueNegativeTestJSON-190719564-project] 10.222.0.26 "GET /compute/v2.1/servers/3da61834-fd67-49d2-925c-c18fc264042c" status: 404 len: 111 microversion: 2.1 time: 0.049706 Oct 19 00:46:09.234865 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 528/643] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:46:09 2022] GET /compute/v2.1/servers/3da61834-fd67-49d2-925c-c18fc264042c => generated 111 bytes in 52 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 19 00:46:09.641787 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-348eeb74-9912-460b-abf4-9def992f6aba tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:46:09.642974 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-348eeb74-9912-460b-abf4-9def992f6aba tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] [instance: d8ff0e88-597f-4db1-b5f4-852dd33c566d] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:46:09.655910 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-348eeb74-9912-460b-abf4-9def992f6aba tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:46:09.656799 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-348eeb74-9912-460b-abf4-9def992f6aba tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:46:09.657607 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-348eeb74-9912-460b-abf4-9def992f6aba tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:46:09.658947 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-348eeb74-9912-460b-abf4-9def992f6aba tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:46:09.659776 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-348eeb74-9912-460b-abf4-9def992f6aba tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:46:09.660571 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-348eeb74-9912-460b-abf4-9def992f6aba tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:46:09.693742 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.wsgi [None req-348eeb74-9912-460b-abf4-9def992f6aba tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] HTTP exception thrown: Instance d8ff0e88-597f-4db1-b5f4-852dd33c566d could not be found. Oct 19 00:46:09.697090 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-348eeb74-9912-460b-abf4-9def992f6aba tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Request method failure captured: Oct 19 00:46:09.697090 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: request: GET /compute/v2.1/servers/d8ff0e88-597f-4db1-b5f4-852dd33c566d HTTP/1.1 Oct 19 00:46:09.697090 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Accept: application/json Oct 19 00:46:09.697090 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Accept-Encoding: identity Oct 19 00:46:09.697090 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Connection: close Oct 19 00:46:09.697090 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Content-Length: 0 Oct 19 00:46:09.697090 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Content-Type: application/json Oct 19 00:46:09.697090 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Host: 10.222.0.26 Oct 19 00:46:09.697090 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Openstack-System-Scope: None Oct 19 00:46:09.697090 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: User-Agent: python-urllib3/1.26.12 Oct 19 00:46:09.697090 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Auth-Token: gAAAAABjT0eJXd9ar1txWHw929SSSa4BnzN-f-jxcCDsiqb5Qu6eweguqGUJXghFM8xaTx1ciC3SJFoac87bZHlU5VANYjjzV6ICQChedig2cB1RmNul9h3AYlunCOiAit1Pl73UlpO5HCAxVXGmz0N3G59VsZacZSMYF6RytJzlde2rg7lCpks Oct 19 00:46:09.697090 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Domain-Id: None Oct 19 00:46:09.697090 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Domain-Name: None Oct 19 00:46:09.697090 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Identity-Status: Confirmed Oct 19 00:46:09.697090 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Is-Admin-Project: True Oct 19 00:46:09.697090 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Project-Domain-Id: default Oct 19 00:46:09.697090 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Project-Domain-Name: Default Oct 19 00:46:09.697090 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Project-Id: 0b35c2038f6b4d0a9012b004f1416c00 Oct 19 00:46:09.697090 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Project-Name: tempest-AttachVolumeNegativeTest-1282551484 Oct 19 00:46:09.697090 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Role: member,reader Oct 19 00:46:09.697090 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Roles: member,reader Oct 19 00:46:09.700236 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/0b35c2038f6b4d0a9012b004f1416c00"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/0b35c2038f6b4d0a9012b004f1416c00"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/0b35c2038f6b4d0a9012b004f1416c00"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_0b35c2038f6b4d0a9012b004f1416c00"}]}] Oct 19 00:46:09.700236 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Tenant: tempest-AttachVolumeNegativeTest-1282551484 Oct 19 00:46:09.700236 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Tenant-Id: 0b35c2038f6b4d0a9012b004f1416c00 Oct 19 00:46:09.700236 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1282551484 Oct 19 00:46:09.700236 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User: tempest-AttachVolumeNegativeTest-1282551484-project Oct 19 00:46:09.700236 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User-Domain-Id: default Oct 19 00:46:09.700236 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User-Domain-Name: Default Oct 19 00:46:09.700236 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User-Id: 6f8725c4d05b4dd79b05b43fba15980b Oct 19 00:46:09.700236 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User-Name: tempest-AttachVolumeNegativeTest-1282551484-project Oct 19 00:46:09.700236 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: method: > Oct 19 00:46:09.700236 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: exception: Instance d8ff0e88-597f-4db1-b5f4-852dd33c566d could not be found. Oct 19 00:46:09.700236 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: action_args: {'id': 'd8ff0e88-597f-4db1-b5f4-852dd33c566d'} Oct 19 00:46:09.702550 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 19 00:46:09.702550 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 19 00:46:09.702550 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 19 00:46:09.702550 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 19 00:46:09.702550 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 19 00:46:09.702550 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 19 00:46:09.702550 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 19 00:46:09.702550 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 19 00:46:09.702550 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 19 00:46:09.702550 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 19 00:46:09.702550 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance d8ff0e88-597f-4db1-b5f4-852dd33c566d could not be found. Oct 19 00:46:09.702550 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:46:09.702550 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 19 00:46:09.702550 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:46:09.702550 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 19 00:46:09.702550 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 19 00:46:09.702550 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 19 00:46:09.702550 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 19 00:46:09.702550 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 19 00:46:09.702550 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 19 00:46:09.705157 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 19 00:46:09.705157 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 19 00:46:09.705157 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 19 00:46:09.705157 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 19 00:46:09.705157 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 19 00:46:09.705157 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 19 00:46:09.705157 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 19 00:46:09.705157 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance d8ff0e88-597f-4db1-b5f4-852dd33c566d could not be found. Oct 19 00:46:09.705157 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:46:09.705157 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 19 00:46:09.705157 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:46:09.705157 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 19 00:46:09.705157 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 19 00:46:09.705157 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 19 00:46:09.705157 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 19 00:46:09.705157 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 19 00:46:09.705157 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance d8ff0e88-597f-4db1-b5f4-852dd33c566d could not be found. Oct 19 00:46:09.705157 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:46:09.710107 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-348eeb74-9912-460b-abf4-9def992f6aba tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Returning 404 to user: Instance d8ff0e88-597f-4db1-b5f4-852dd33c566d could not be found. {{(pid=125641) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 19 00:46:09.710107 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-348eeb74-9912-460b-abf4-9def992f6aba tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] 10.222.0.26 "GET /compute/v2.1/servers/d8ff0e88-597f-4db1-b5f4-852dd33c566d" status: 404 len: 111 microversion: 2.1 time: 0.065153 Oct 19 00:46:09.710107 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 529/644] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:46:09 2022] GET /compute/v2.1/servers/d8ff0e88-597f-4db1-b5f4-852dd33c566d => generated 111 bytes in 68 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Oct 19 00:46:09.714450 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-97b26f17-e14a-4933-9ff7-3faf567e6bd2 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Calling method '.version_select at 0x7f71c3f30af0>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:46:09.744470 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-97b26f17-e14a-4933-9ff7-3faf567e6bd2 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] 10.222.0.26 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-1805314021" status: 202 len: 0 microversion: 2.1 time: 0.036515 Oct 19 00:46:09.745737 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 530/645] 10.222.0.26 () {58 vars in 1221 bytes} [Wed Oct 19 00:46:09 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-1805314021 => generated 0 bytes in 39 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Oct 19 00:46:09.988858 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [req-31e52051-0641-47b2-b77d-14540da4f05a req-a6ccdce5-017c-45bb-a87d-0479cf520f95 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "d8ff0e88-597f-4db1-b5f4-852dd33c566d", "name": "network-vif-deleted", "tag": "1598d129-803e-437a-be2e-c9127e38bd88"}]} {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 19 00:46:10.022858 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-31e52051-0641-47b2-b77d-14540da4f05a req-a6ccdce5-017c-45bb-a87d-0479cf520f95 service nova] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:46:10.024663 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-31e52051-0641-47b2-b77d-14540da4f05a req-a6ccdce5-017c-45bb-a87d-0479cf520f95 service nova] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:46:10.025520 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-31e52051-0641-47b2-b77d-14540da4f05a req-a6ccdce5-017c-45bb-a87d-0479cf520f95 service nova] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:46:10.060621 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.compute.server_external_events [req-31e52051-0641-47b2-b77d-14540da4f05a req-a6ccdce5-017c-45bb-a87d-0479cf520f95 service nova] Creating event network-vif-deleted:1598d129-803e-437a-be2e-c9127e38bd88 for instance d8ff0e88-597f-4db1-b5f4-852dd33c566d on hv-cinder-maste Oct 19 00:46:10.068489 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [req-31e52051-0641-47b2-b77d-14540da4f05a req-a6ccdce5-017c-45bb-a87d-0479cf520f95 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.086897 Oct 19 00:46:10.069667 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 531/646] 10.222.0.26 () {64 vars in 1317 bytes} [Wed Oct 19 00:46:09 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 19 00:46:10.957979 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-33bec0db-809e-4c5d-ba0f-148769fa1733 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Action: 'create', calling method: .version_select at 0x7f71cb58b700>, body: {"keypair": {"name": "tempest-keypair-1023768314", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBIhkbt/SSW26b3TvrUqRUI7ROf3disOyvXo2Do6Mja8Bez9VDOQyfJxAo7qTRNGemqDV0y0+NVjd9IFE9mWIK05EBC0rMnvRVwOoGy85ScX3VXieSbeO071ThA2UK22jnQ=="}} {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 19 00:46:10.986677 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.quota [None req-33bec0db-809e-4c5d-ba0f-148769fa1733 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Getting quotas for project 0b35c2038f6b4d0a9012b004f1416c00. Resources: {'key_pairs'} {{(pid=125641) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 19 00:46:10.994389 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.quota [None req-33bec0db-809e-4c5d-ba0f-148769fa1733 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Getting quotas for user 6f8725c4d05b4dd79b05b43fba15980b and project 0b35c2038f6b4d0a9012b004f1416c00. Resources: {'key_pairs'} {{(pid=125641) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 19 00:46:11.015705 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG dbcounter [-] [125641] Writing DB stats nova_api:SELECT=96,nova_api:UPDATE=3,nova_api:DELETE=1 {{(pid=125641) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 00:46:11.043012 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-33bec0db-809e-4c5d-ba0f-148769fa1733 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] 10.222.0.26 "POST /compute/v2.1/os-keypairs" status: 200 len: 386 microversion: 2.1 time: 0.093455 Oct 19 00:46:11.051100 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 532/647] 10.222.0.26 () {60 vars in 1159 bytes} [Wed Oct 19 00:46:10 2022] POST /compute/v2.1/os-keypairs => generated 386 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 19 00:46:13.417461 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-6f629829-0859-491c-870d-9e3ca4c870b4 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeNegativeTest-server-168190964", "imageRef": "b387b4f8-f75e-4a68-b49c-a015bf61c46b", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "635fade4-7ee9-45f4-8020-ceb7d6bf784a"}], "security_groups": [{"name": "tempest-securitygroup--1789392836"}], "key_name": "tempest-keypair-1023768314", "user_data": "}} {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 19 00:46:13.675059 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.network.neutron [None req-6f629829-0859-491c-870d-9e3ca4c870b4 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] validate_networks() for [('635fade4-7ee9-45f4-8020-ceb7d6bf784a', None, None, None, None, None)] {{(pid=125641) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 19 00:46:14.605419 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.virt.hardware [None req-6f629829-0859-491c-870d-9e3ca4c870b4 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Flavor limits 0:0:0 {{(pid=125641) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 19 00:46:14.606328 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.virt.hardware [None req-6f629829-0859-491c-870d-9e3ca4c870b4 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Image limits 0:0:0 {{(pid=125641) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 19 00:46:14.607104 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.virt.hardware [None req-6f629829-0859-491c-870d-9e3ca4c870b4 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Flavor pref 0:0:0 {{(pid=125641) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 19 00:46:14.607815 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.virt.hardware [None req-6f629829-0859-491c-870d-9e3ca4c870b4 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Image pref 0:0:0 {{(pid=125641) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 19 00:46:14.608367 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.virt.hardware [None req-6f629829-0859-491c-870d-9e3ca4c870b4 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=125641) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 19 00:46:14.624690 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-6f629829-0859-491c-870d-9e3ca4c870b4 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:46:14.625326 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-6f629829-0859-491c-870d-9e3ca4c870b4 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:46:14.625909 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-6f629829-0859-491c-870d-9e3ca4c870b4 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:46:14.626834 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-6f629829-0859-491c-870d-9e3ca4c870b4 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:46:14.627402 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-6f629829-0859-491c-870d-9e3ca4c870b4 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:46:14.627942 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-6f629829-0859-491c-870d-9e3ca4c870b4 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:46:14.655215 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.quota [None req-6f629829-0859-491c-870d-9e3ca4c870b4 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Getting quotas for project 0b35c2038f6b4d0a9012b004f1416c00. Resources: {'cores', 'ram', 'instances'} {{(pid=125641) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 19 00:46:14.660061 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.quota [None req-6f629829-0859-491c-870d-9e3ca4c870b4 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Getting quotas for user 6f8725c4d05b4dd79b05b43fba15980b and project 0b35c2038f6b4d0a9012b004f1416c00. Resources: {'cores', 'ram', 'instances'} {{(pid=125641) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 19 00:46:14.673918 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-6f629829-0859-491c-870d-9e3ca4c870b4 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Going to run 1 instances... {{(pid=125641) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 19 00:46:14.676475 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-6f629829-0859-491c-870d-9e3ca4c870b4 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] [instance: 0da74d88-6672-4df0-b665-b9eec2d073c5] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='b387b4f8-f75e-4a68-b49c-a015bf61c46b',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=125641) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 19 00:46:14.745547 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-6f629829-0859-491c-870d-9e3ca4c870b4 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 202 len: 402 microversion: 2.1 time: 1.337225 Oct 19 00:46:14.753393 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 533/648] 10.222.0.26 () {60 vars in 1147 bytes} [Wed Oct 19 00:46:13 2022] POST /compute/v2.1/servers => generated 402 bytes in 1341 msecs (HTTP/1.1 202) 10 headers in 451 bytes (2 switches on core 0) Oct 19 00:46:14.763003 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-e738ab95-c162-4878-a4e9-bd64dce60fa5 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:46:14.764103 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-e738ab95-c162-4878-a4e9-bd64dce60fa5 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] [instance: 0da74d88-6672-4df0-b665-b9eec2d073c5] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:46:14.863201 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-e738ab95-c162-4878-a4e9-bd64dce60fa5 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f8725c4d05b4dd79b05b43fba15980b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0b35c2038f6b4d0a9012b004f1416c00', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:46:14.875445 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-e738ab95-c162-4878-a4e9-bd64dce60fa5 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] 10.222.0.26 "GET /compute/v2.1/servers/0da74d88-6672-4df0-b665-b9eec2d073c5" status: 200 len: 1241 microversion: 2.1 time: 0.119941 Oct 19 00:46:14.876604 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 534/649] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:46:14 2022] GET /compute/v2.1/servers/0da74d88-6672-4df0-b665-b9eec2d073c5 => generated 1241 bytes in 122 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 19 00:46:15.305069 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-2c156e30-3c6a-429b-8a56-d6e88da905a7 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:46:15.308102 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-2c156e30-3c6a-429b-8a56-d6e88da905a7 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] [instance: 85efeeae-0648-4a33-bc74-c0f0d25880aa] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:46:15.327708 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-2c156e30-3c6a-429b-8a56-d6e88da905a7 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:46:15.328500 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-2c156e30-3c6a-429b-8a56-d6e88da905a7 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:46:15.329007 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-2c156e30-3c6a-429b-8a56-d6e88da905a7 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:46:15.330005 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-2c156e30-3c6a-429b-8a56-d6e88da905a7 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:46:15.330647 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-2c156e30-3c6a-429b-8a56-d6e88da905a7 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:46:15.331421 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-2c156e30-3c6a-429b-8a56-d6e88da905a7 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:46:15.380457 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG dbcounter [-] [125641] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=125641) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 00:46:15.390286 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-2c156e30-3c6a-429b-8a56-d6e88da905a7 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] 10.222.0.26 "GET /compute/v2.1/servers/85efeeae-0648-4a33-bc74-c0f0d25880aa/ips" status: 200 len: 151 microversion: 2.20 time: 0.427529 Oct 19 00:46:15.391128 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 535/650] 10.222.0.26 () {60 vars in 1289 bytes} [Wed Oct 19 00:46:14 2022] GET /compute/v2.1/servers/85efeeae-0648-4a33-bc74-c0f0d25880aa/ips => generated 151 bytes in 431 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Oct 19 00:46:15.907165 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-1650d186-38a8-456e-bfbd-892c3ffa08fe tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:46:15.908301 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-1650d186-38a8-456e-bfbd-892c3ffa08fe tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] [instance: 0da74d88-6672-4df0-b665-b9eec2d073c5] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:46:15.937044 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-1650d186-38a8-456e-bfbd-892c3ffa08fe tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:46:15.939075 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-1650d186-38a8-456e-bfbd-892c3ffa08fe tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:46:15.940323 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-1650d186-38a8-456e-bfbd-892c3ffa08fe tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:46:15.942120 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-1650d186-38a8-456e-bfbd-892c3ffa08fe tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:46:15.943536 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-1650d186-38a8-456e-bfbd-892c3ffa08fe tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:46:15.944397 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-1650d186-38a8-456e-bfbd-892c3ffa08fe tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:46:16.058905 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-1650d186-38a8-456e-bfbd-892c3ffa08fe tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f8725c4d05b4dd79b05b43fba15980b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0b35c2038f6b4d0a9012b004f1416c00', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:46:16.077049 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-1650d186-38a8-456e-bfbd-892c3ffa08fe tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] 10.222.0.26 "GET /compute/v2.1/servers/0da74d88-6672-4df0-b665-b9eec2d073c5" status: 200 len: 1245 microversion: 2.1 time: 0.177207 Oct 19 00:46:16.078673 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 536/651] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:46:15 2022] GET /compute/v2.1/servers/0da74d88-6672-4df0-b665-b9eec2d073c5 => generated 1245 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 19 00:46:17.117043 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-13ddc995-cd9a-4cd4-b370-3038aa0513b6 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:46:17.117902 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-13ddc995-cd9a-4cd4-b370-3038aa0513b6 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] [instance: 0da74d88-6672-4df0-b665-b9eec2d073c5] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:46:17.132049 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-13ddc995-cd9a-4cd4-b370-3038aa0513b6 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:46:17.133019 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-13ddc995-cd9a-4cd4-b370-3038aa0513b6 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:46:17.133626 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-13ddc995-cd9a-4cd4-b370-3038aa0513b6 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:46:17.134751 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-13ddc995-cd9a-4cd4-b370-3038aa0513b6 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:46:17.135362 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-13ddc995-cd9a-4cd4-b370-3038aa0513b6 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:46:17.136011 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-13ddc995-cd9a-4cd4-b370-3038aa0513b6 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:46:17.227360 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-13ddc995-cd9a-4cd4-b370-3038aa0513b6 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f8725c4d05b4dd79b05b43fba15980b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0b35c2038f6b4d0a9012b004f1416c00', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:46:17.236850 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-13ddc995-cd9a-4cd4-b370-3038aa0513b6 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] 10.222.0.26 "GET /compute/v2.1/servers/0da74d88-6672-4df0-b665-b9eec2d073c5" status: 200 len: 1299 microversion: 2.1 time: 0.127539 Oct 19 00:46:17.245104 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 537/652] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:46:17 2022] GET /compute/v2.1/servers/0da74d88-6672-4df0-b665-b9eec2d073c5 => generated 1299 bytes in 131 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 19 00:46:17.574468 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-0ccc1d77-5e57-4302-a982-35d5539b8c22 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Action: 'action', calling method: >, body: {"shelve": {}} {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 19 00:46:17.575480 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-0ccc1d77-5e57-4302-a982-35d5539b8c22 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] [instance: 85efeeae-0648-4a33-bc74-c0f0d25880aa] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:46:17.587424 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-0ccc1d77-5e57-4302-a982-35d5539b8c22 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:46:17.588743 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-0ccc1d77-5e57-4302-a982-35d5539b8c22 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:46:17.589823 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-0ccc1d77-5e57-4302-a982-35d5539b8c22 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:46:17.591404 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-0ccc1d77-5e57-4302-a982-35d5539b8c22 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:46:17.592510 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-0ccc1d77-5e57-4302-a982-35d5539b8c22 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:46:17.593632 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-0ccc1d77-5e57-4302-a982-35d5539b8c22 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:46:17.645345 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.objects.instance [None req-0ccc1d77-5e57-4302-a982-35d5539b8c22 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lazy-loading 'flavor' on Instance uuid 85efeeae-0648-4a33-bc74-c0f0d25880aa {{(pid=125641) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 19 00:46:17.906414 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-0ccc1d77-5e57-4302-a982-35d5539b8c22 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] 10.222.0.26 "POST /compute/v2.1/servers/85efeeae-0648-4a33-bc74-c0f0d25880aa/action" status: 202 len: 0 microversion: 2.20 time: 0.366811 Oct 19 00:46:17.907718 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 538/653] 10.222.0.26 () {62 vars in 1319 bytes} [Wed Oct 19 00:46:17 2022] POST /compute/v2.1/servers/85efeeae-0648-4a33-bc74-c0f0d25880aa/action => generated 0 bytes in 370 msecs (HTTP/1.1 202) 9 headers in 363 bytes (1 switches on core 0) Oct 19 00:46:17.922298 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-197155bd-7633-466e-9757-701b515d90b1 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:46:17.924468 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-197155bd-7633-466e-9757-701b515d90b1 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] [instance: 85efeeae-0648-4a33-bc74-c0f0d25880aa] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:46:17.941943 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-197155bd-7633-466e-9757-701b515d90b1 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:46:17.943622 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-197155bd-7633-466e-9757-701b515d90b1 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:46:17.945390 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-197155bd-7633-466e-9757-701b515d90b1 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:46:17.947115 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-197155bd-7633-466e-9757-701b515d90b1 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:46:17.948074 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-197155bd-7633-466e-9757-701b515d90b1 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:46:17.949171 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-197155bd-7633-466e-9757-701b515d90b1 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:46:18.171890 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-197155bd-7633-466e-9757-701b515d90b1 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:46:18.184824 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-197155bd-7633-466e-9757-701b515d90b1 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:46:18.186339 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-197155bd-7633-466e-9757-701b515d90b1 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:46:18.187719 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-197155bd-7633-466e-9757-701b515d90b1 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] 10.222.0.26 "GET /compute/v2.1/servers/85efeeae-0648-4a33-bc74-c0f0d25880aa" status: 200 len: 1723 microversion: 2.20 time: 0.273899 Oct 19 00:46:18.188768 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 539/654] 10.222.0.26 () {60 vars in 1277 bytes} [Wed Oct 19 00:46:17 2022] GET /compute/v2.1/servers/85efeeae-0648-4a33-bc74-c0f0d25880aa => generated 1723 bytes in 276 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 19 00:46:18.263612 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-05540d86-f0ca-4be1-a006-6b68af8e8b18 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:46:18.264553 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-05540d86-f0ca-4be1-a006-6b68af8e8b18 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] [instance: 0da74d88-6672-4df0-b665-b9eec2d073c5] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:46:18.271915 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-05540d86-f0ca-4be1-a006-6b68af8e8b18 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:46:18.273583 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-05540d86-f0ca-4be1-a006-6b68af8e8b18 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:46:18.274225 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-05540d86-f0ca-4be1-a006-6b68af8e8b18 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:46:18.275212 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-05540d86-f0ca-4be1-a006-6b68af8e8b18 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:46:18.275885 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-05540d86-f0ca-4be1-a006-6b68af8e8b18 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:46:18.276529 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-05540d86-f0ca-4be1-a006-6b68af8e8b18 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:46:18.916518 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-05540d86-f0ca-4be1-a006-6b68af8e8b18 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f8725c4d05b4dd79b05b43fba15980b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0b35c2038f6b4d0a9012b004f1416c00', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:46:18.932576 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-05540d86-f0ca-4be1-a006-6b68af8e8b18 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] 10.222.0.26 "GET /compute/v2.1/servers/0da74d88-6672-4df0-b665-b9eec2d073c5" status: 200 len: 1367 microversion: 2.1 time: 0.673176 Oct 19 00:46:18.935701 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 540/655] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:46:18 2022] GET /compute/v2.1/servers/0da74d88-6672-4df0-b665-b9eec2d073c5 => generated 1367 bytes in 676 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:46:19.220633 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-40a98716-6050-48a7-9340-a5d13294e90a tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:46:19.223139 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-40a98716-6050-48a7-9340-a5d13294e90a tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] [instance: 85efeeae-0648-4a33-bc74-c0f0d25880aa] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:46:19.237598 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-40a98716-6050-48a7-9340-a5d13294e90a tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:46:19.239344 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-40a98716-6050-48a7-9340-a5d13294e90a tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:46:19.240626 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-40a98716-6050-48a7-9340-a5d13294e90a tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:46:19.242400 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-40a98716-6050-48a7-9340-a5d13294e90a tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:46:19.243739 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-40a98716-6050-48a7-9340-a5d13294e90a tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:46:19.246146 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-40a98716-6050-48a7-9340-a5d13294e90a tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:46:19.671121 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-40a98716-6050-48a7-9340-a5d13294e90a tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:46:19.694733 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-40a98716-6050-48a7-9340-a5d13294e90a tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:46:19.698269 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-40a98716-6050-48a7-9340-a5d13294e90a tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:46:19.700318 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-40a98716-6050-48a7-9340-a5d13294e90a tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] 10.222.0.26 "GET /compute/v2.1/servers/85efeeae-0648-4a33-bc74-c0f0d25880aa" status: 200 len: 1723 microversion: 2.20 time: 0.487503 Oct 19 00:46:19.701356 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 541/656] 10.222.0.26 () {60 vars in 1277 bytes} [Wed Oct 19 00:46:19 2022] GET /compute/v2.1/servers/85efeeae-0648-4a33-bc74-c0f0d25880aa => generated 1723 bytes in 490 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 19 00:46:19.715736 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [req-b150a0b0-0bd6-4042-9d2f-3e086e52e8df req-70a7ec9e-a8ef-44c9-9556-32091e0ba7f8 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "0da74d88-6672-4df0-b665-b9eec2d073c5", "tag": "6ccabad6-2670-419e-95ab-c0c5d0749b3a"}]} {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 19 00:46:19.740115 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-b150a0b0-0bd6-4042-9d2f-3e086e52e8df req-70a7ec9e-a8ef-44c9-9556-32091e0ba7f8 service nova] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:46:19.741559 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-b150a0b0-0bd6-4042-9d2f-3e086e52e8df req-70a7ec9e-a8ef-44c9-9556-32091e0ba7f8 service nova] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:46:19.742274 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-b150a0b0-0bd6-4042-9d2f-3e086e52e8df req-70a7ec9e-a8ef-44c9-9556-32091e0ba7f8 service nova] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:46:19.761403 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.compute.server_external_events [req-b150a0b0-0bd6-4042-9d2f-3e086e52e8df req-70a7ec9e-a8ef-44c9-9556-32091e0ba7f8 service nova] Creating event network-changed:6ccabad6-2670-419e-95ab-c0c5d0749b3a for instance 0da74d88-6672-4df0-b665-b9eec2d073c5 on hv-cinder-maste Oct 19 00:46:19.771583 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [req-b150a0b0-0bd6-4042-9d2f-3e086e52e8df req-70a7ec9e-a8ef-44c9-9556-32091e0ba7f8 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.067483 Oct 19 00:46:19.779344 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 542/657] 10.222.0.26 () {64 vars in 1317 bytes} [Wed Oct 19 00:46:19 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 77 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 19 00:46:19.977958 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-1cd20ce7-fdaa-4dd6-a38b-c579f089f5b7 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:46:19.978837 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-1cd20ce7-fdaa-4dd6-a38b-c579f089f5b7 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] [instance: 0da74d88-6672-4df0-b665-b9eec2d073c5] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:46:19.989973 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-1cd20ce7-fdaa-4dd6-a38b-c579f089f5b7 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:46:19.991194 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-1cd20ce7-fdaa-4dd6-a38b-c579f089f5b7 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:46:19.991738 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-1cd20ce7-fdaa-4dd6-a38b-c579f089f5b7 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:46:19.992569 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-1cd20ce7-fdaa-4dd6-a38b-c579f089f5b7 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:46:19.993162 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-1cd20ce7-fdaa-4dd6-a38b-c579f089f5b7 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:46:19.993670 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-1cd20ce7-fdaa-4dd6-a38b-c579f089f5b7 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:46:20.263575 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-1cd20ce7-fdaa-4dd6-a38b-c579f089f5b7 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f8725c4d05b4dd79b05b43fba15980b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0b35c2038f6b4d0a9012b004f1416c00', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:46:20.275745 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-1cd20ce7-fdaa-4dd6-a38b-c579f089f5b7 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] 10.222.0.26 "GET /compute/v2.1/servers/0da74d88-6672-4df0-b665-b9eec2d073c5" status: 200 len: 1367 microversion: 2.1 time: 0.302445 Oct 19 00:46:20.276639 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 543/658] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:46:19 2022] GET /compute/v2.1/servers/0da74d88-6672-4df0-b665-b9eec2d073c5 => generated 1367 bytes in 305 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 19 00:46:20.733309 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-ebe2f562-c5f2-4347-8bfc-a1d5a8aeeda2 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:46:20.734286 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-ebe2f562-c5f2-4347-8bfc-a1d5a8aeeda2 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] [instance: 85efeeae-0648-4a33-bc74-c0f0d25880aa] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:46:20.767421 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ebe2f562-c5f2-4347-8bfc-a1d5a8aeeda2 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:46:20.768224 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ebe2f562-c5f2-4347-8bfc-a1d5a8aeeda2 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:46:20.768826 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ebe2f562-c5f2-4347-8bfc-a1d5a8aeeda2 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:46:20.769839 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ebe2f562-c5f2-4347-8bfc-a1d5a8aeeda2 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:46:20.770673 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ebe2f562-c5f2-4347-8bfc-a1d5a8aeeda2 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:46:20.771186 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ebe2f562-c5f2-4347-8bfc-a1d5a8aeeda2 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:46:21.016521 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-ebe2f562-c5f2-4347-8bfc-a1d5a8aeeda2 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:46:21.026391 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-ebe2f562-c5f2-4347-8bfc-a1d5a8aeeda2 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:46:21.028498 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-ebe2f562-c5f2-4347-8bfc-a1d5a8aeeda2 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:46:21.030163 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-ebe2f562-c5f2-4347-8bfc-a1d5a8aeeda2 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] 10.222.0.26 "GET /compute/v2.1/servers/85efeeae-0648-4a33-bc74-c0f0d25880aa" status: 200 len: 1723 microversion: 2.20 time: 0.305863 Oct 19 00:46:21.030853 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 544/659] 10.222.0.26 () {60 vars in 1277 bytes} [Wed Oct 19 00:46:20 2022] GET /compute/v2.1/servers/85efeeae-0648-4a33-bc74-c0f0d25880aa => generated 1723 bytes in 308 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 19 00:46:21.324378 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-8ac86538-c178-49d9-a461-30eafdc0f470 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:46:21.327053 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-8ac86538-c178-49d9-a461-30eafdc0f470 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] [instance: 0da74d88-6672-4df0-b665-b9eec2d073c5] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:46:21.343509 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-8ac86538-c178-49d9-a461-30eafdc0f470 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:46:21.346526 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-8ac86538-c178-49d9-a461-30eafdc0f470 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:46:21.348325 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-8ac86538-c178-49d9-a461-30eafdc0f470 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:46:21.349987 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-8ac86538-c178-49d9-a461-30eafdc0f470 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:46:21.351178 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-8ac86538-c178-49d9-a461-30eafdc0f470 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:46:21.352315 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-8ac86538-c178-49d9-a461-30eafdc0f470 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:46:21.649980 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-8ac86538-c178-49d9-a461-30eafdc0f470 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f8725c4d05b4dd79b05b43fba15980b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0b35c2038f6b4d0a9012b004f1416c00', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:46:21.661861 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-8ac86538-c178-49d9-a461-30eafdc0f470 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] 10.222.0.26 "GET /compute/v2.1/servers/0da74d88-6672-4df0-b665-b9eec2d073c5" status: 200 len: 1367 microversion: 2.1 time: 0.351356 Oct 19 00:46:21.663767 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 545/660] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:46:21 2022] GET /compute/v2.1/servers/0da74d88-6672-4df0-b665-b9eec2d073c5 => generated 1367 bytes in 354 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 19 00:46:22.082470 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-5da3641f-ae6f-43e7-88e0-855ef7f821e4 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:46:22.083343 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-5da3641f-ae6f-43e7-88e0-855ef7f821e4 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] [instance: 85efeeae-0648-4a33-bc74-c0f0d25880aa] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:46:22.104216 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-5da3641f-ae6f-43e7-88e0-855ef7f821e4 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:46:22.104859 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-5da3641f-ae6f-43e7-88e0-855ef7f821e4 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:46:22.105920 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-5da3641f-ae6f-43e7-88e0-855ef7f821e4 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:46:22.106746 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-5da3641f-ae6f-43e7-88e0-855ef7f821e4 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:46:22.107188 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-5da3641f-ae6f-43e7-88e0-855ef7f821e4 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:46:22.108323 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-5da3641f-ae6f-43e7-88e0-855ef7f821e4 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:46:22.375024 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-5da3641f-ae6f-43e7-88e0-855ef7f821e4 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:46:22.385031 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-5da3641f-ae6f-43e7-88e0-855ef7f821e4 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:46:22.386716 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-5da3641f-ae6f-43e7-88e0-855ef7f821e4 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:46:22.388027 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-5da3641f-ae6f-43e7-88e0-855ef7f821e4 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] 10.222.0.26 "GET /compute/v2.1/servers/85efeeae-0648-4a33-bc74-c0f0d25880aa" status: 200 len: 1723 microversion: 2.20 time: 0.310777 Oct 19 00:46:22.389847 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 546/661] 10.222.0.26 () {60 vars in 1277 bytes} [Wed Oct 19 00:46:22 2022] GET /compute/v2.1/servers/85efeeae-0648-4a33-bc74-c0f0d25880aa => generated 1723 bytes in 312 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 19 00:46:22.701995 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-3a31e831-aff0-4f51-a758-b65e0871e418 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:46:22.703085 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-3a31e831-aff0-4f51-a758-b65e0871e418 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] [instance: 0da74d88-6672-4df0-b665-b9eec2d073c5] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:46:22.713108 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-3a31e831-aff0-4f51-a758-b65e0871e418 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:46:22.714384 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-3a31e831-aff0-4f51-a758-b65e0871e418 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:46:22.715193 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-3a31e831-aff0-4f51-a758-b65e0871e418 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:46:22.716701 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-3a31e831-aff0-4f51-a758-b65e0871e418 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:46:22.717535 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-3a31e831-aff0-4f51-a758-b65e0871e418 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:46:22.718143 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-3a31e831-aff0-4f51-a758-b65e0871e418 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:46:22.983841 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-3a31e831-aff0-4f51-a758-b65e0871e418 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f8725c4d05b4dd79b05b43fba15980b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0b35c2038f6b4d0a9012b004f1416c00', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:46:22.998958 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-3a31e831-aff0-4f51-a758-b65e0871e418 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] 10.222.0.26 "GET /compute/v2.1/servers/0da74d88-6672-4df0-b665-b9eec2d073c5" status: 200 len: 1367 microversion: 2.1 time: 0.303246 Oct 19 00:46:23.000452 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 547/662] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:46:22 2022] GET /compute/v2.1/servers/0da74d88-6672-4df0-b665-b9eec2d073c5 => generated 1367 bytes in 307 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 19 00:46:23.438139 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-49678267-5ab1-4aff-99ae-d1a3d98ecb15 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:46:23.439335 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-49678267-5ab1-4aff-99ae-d1a3d98ecb15 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] [instance: 85efeeae-0648-4a33-bc74-c0f0d25880aa] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:46:23.462083 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-49678267-5ab1-4aff-99ae-d1a3d98ecb15 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:46:23.463332 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-49678267-5ab1-4aff-99ae-d1a3d98ecb15 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:46:23.464345 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-49678267-5ab1-4aff-99ae-d1a3d98ecb15 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:46:23.465780 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-49678267-5ab1-4aff-99ae-d1a3d98ecb15 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:46:23.466877 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-49678267-5ab1-4aff-99ae-d1a3d98ecb15 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:46:23.467898 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-49678267-5ab1-4aff-99ae-d1a3d98ecb15 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:46:23.784680 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-49678267-5ab1-4aff-99ae-d1a3d98ecb15 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:46:23.804418 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-49678267-5ab1-4aff-99ae-d1a3d98ecb15 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:46:23.806376 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-49678267-5ab1-4aff-99ae-d1a3d98ecb15 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:46:23.807829 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-49678267-5ab1-4aff-99ae-d1a3d98ecb15 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] 10.222.0.26 "GET /compute/v2.1/servers/85efeeae-0648-4a33-bc74-c0f0d25880aa" status: 200 len: 1723 microversion: 2.20 time: 0.377442 Oct 19 00:46:23.808782 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 548/663] 10.222.0.26 () {60 vars in 1277 bytes} [Wed Oct 19 00:46:23 2022] GET /compute/v2.1/servers/85efeeae-0648-4a33-bc74-c0f0d25880aa => generated 1723 bytes in 379 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 19 00:46:24.190585 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-619dea44-c2c2-4edd-8498-ff5492304ffa tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Action: 'create', calling method: .version_select at 0x7f71cb53cdc0>, body: {"keypair": {"name": "tempest-keypair-1137457954", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBDSY3Wy75r4N/KF0Mt+qB2uMc74h3r/ivQT1hfM5LJ+nLT24AloQjk9zN6vp/0H+8N6FKAj4nYGy+4ypu23iaz6wfdrKjW8SDfWdL6QtpgVJXn+5WcqhDQUiGWg1cNWXcQ=="}} {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 19 00:46:24.210298 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.quota [None req-619dea44-c2c2-4edd-8498-ff5492304ffa tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Getting quotas for project 82d7723ef95b4c3faeac9835eeb233a5. Resources: {'key_pairs'} {{(pid=125641) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 19 00:46:24.216281 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.quota [None req-619dea44-c2c2-4edd-8498-ff5492304ffa tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Getting quotas for user a48ed2d9ded741ca91afc1fbbd9c608f and project 82d7723ef95b4c3faeac9835eeb233a5. Resources: {'key_pairs'} {{(pid=125641) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 19 00:46:24.248390 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-619dea44-c2c2-4edd-8498-ff5492304ffa tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] 10.222.0.26 "POST /compute/v2.1/os-keypairs" status: 200 len: 386 microversion: 2.1 time: 0.236495 Oct 19 00:46:24.249037 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 549/664] 10.222.0.26 () {60 vars in 1159 bytes} [Wed Oct 19 00:46:24 2022] POST /compute/v2.1/os-keypairs => generated 386 bytes in 238 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 19 00:46:24.259391 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-1195c73c-7550-4512-a544-213b750956d5 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:46:24.259391 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-1195c73c-7550-4512-a544-213b750956d5 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] [instance: 0da74d88-6672-4df0-b665-b9eec2d073c5] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:46:24.267276 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-1195c73c-7550-4512-a544-213b750956d5 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:46:24.268014 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-1195c73c-7550-4512-a544-213b750956d5 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:46:24.268695 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-1195c73c-7550-4512-a544-213b750956d5 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:46:24.269966 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-1195c73c-7550-4512-a544-213b750956d5 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:46:24.271096 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-1195c73c-7550-4512-a544-213b750956d5 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:46:24.271824 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-1195c73c-7550-4512-a544-213b750956d5 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:46:24.495342 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-1195c73c-7550-4512-a544-213b750956d5 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f8725c4d05b4dd79b05b43fba15980b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0b35c2038f6b4d0a9012b004f1416c00', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:46:24.504949 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-1195c73c-7550-4512-a544-213b750956d5 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] 10.222.0.26 "GET /compute/v2.1/servers/0da74d88-6672-4df0-b665-b9eec2d073c5" status: 200 len: 1367 microversion: 2.1 time: 0.254778 Oct 19 00:46:24.506921 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 550/665] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:46:24 2022] GET /compute/v2.1/servers/0da74d88-6672-4df0-b665-b9eec2d073c5 => generated 1367 bytes in 257 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:46:24.852429 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-d173f821-d4a7-4fff-a93b-5a7ad9ff0836 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:46:24.853964 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-d173f821-d4a7-4fff-a93b-5a7ad9ff0836 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] [instance: 85efeeae-0648-4a33-bc74-c0f0d25880aa] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:46:24.863800 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG dbcounter [-] [125641] Writing DB stats nova_cell0:SELECT=2 {{(pid=125641) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 00:46:24.868631 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d173f821-d4a7-4fff-a93b-5a7ad9ff0836 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:46:24.870448 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d173f821-d4a7-4fff-a93b-5a7ad9ff0836 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:46:24.872325 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d173f821-d4a7-4fff-a93b-5a7ad9ff0836 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:46:24.874005 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d173f821-d4a7-4fff-a93b-5a7ad9ff0836 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:46:24.875181 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d173f821-d4a7-4fff-a93b-5a7ad9ff0836 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:46:24.876204 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d173f821-d4a7-4fff-a93b-5a7ad9ff0836 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:46:25.169730 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-d173f821-d4a7-4fff-a93b-5a7ad9ff0836 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:46:25.187734 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-d173f821-d4a7-4fff-a93b-5a7ad9ff0836 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:46:25.190941 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-d173f821-d4a7-4fff-a93b-5a7ad9ff0836 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:46:25.193227 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-d173f821-d4a7-4fff-a93b-5a7ad9ff0836 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] 10.222.0.26 "GET /compute/v2.1/servers/85efeeae-0648-4a33-bc74-c0f0d25880aa" status: 200 len: 1723 microversion: 2.20 time: 0.349389 Oct 19 00:46:25.195517 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 551/666] 10.222.0.26 () {60 vars in 1277 bytes} [Wed Oct 19 00:46:24 2022] GET /compute/v2.1/servers/85efeeae-0648-4a33-bc74-c0f0d25880aa => generated 1723 bytes in 353 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 19 00:46:25.567688 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-1536ea4a-d235-4ff1-abec-0ee8485ceb54 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:46:25.569428 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-1536ea4a-d235-4ff1-abec-0ee8485ceb54 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] [instance: 0da74d88-6672-4df0-b665-b9eec2d073c5] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:46:25.597330 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-1536ea4a-d235-4ff1-abec-0ee8485ceb54 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:46:25.598695 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-1536ea4a-d235-4ff1-abec-0ee8485ceb54 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:46:25.599873 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-1536ea4a-d235-4ff1-abec-0ee8485ceb54 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:46:25.601237 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-1536ea4a-d235-4ff1-abec-0ee8485ceb54 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:46:25.602102 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-1536ea4a-d235-4ff1-abec-0ee8485ceb54 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:46:25.602945 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-1536ea4a-d235-4ff1-abec-0ee8485ceb54 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:46:25.659793 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG dbcounter [-] [125641] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=125641) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 00:46:25.875800 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-1536ea4a-d235-4ff1-abec-0ee8485ceb54 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f8725c4d05b4dd79b05b43fba15980b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0b35c2038f6b4d0a9012b004f1416c00', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:46:25.889243 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-1536ea4a-d235-4ff1-abec-0ee8485ceb54 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] 10.222.0.26 "GET /compute/v2.1/servers/0da74d88-6672-4df0-b665-b9eec2d073c5" status: 200 len: 1367 microversion: 2.1 time: 0.359380 Oct 19 00:46:25.889971 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 552/667] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:46:25 2022] GET /compute/v2.1/servers/0da74d88-6672-4df0-b665-b9eec2d073c5 => generated 1367 bytes in 362 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 19 00:46:26.242633 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-eb8be498-d715-4a6c-bce4-cffd4ed2eb14 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:46:26.243465 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-eb8be498-d715-4a6c-bce4-cffd4ed2eb14 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] [instance: 85efeeae-0648-4a33-bc74-c0f0d25880aa] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:46:26.249631 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-eb8be498-d715-4a6c-bce4-cffd4ed2eb14 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:46:26.250120 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-eb8be498-d715-4a6c-bce4-cffd4ed2eb14 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:46:26.250684 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-eb8be498-d715-4a6c-bce4-cffd4ed2eb14 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:46:26.251426 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-eb8be498-d715-4a6c-bce4-cffd4ed2eb14 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:46:26.251885 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-eb8be498-d715-4a6c-bce4-cffd4ed2eb14 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:46:26.252313 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-eb8be498-d715-4a6c-bce4-cffd4ed2eb14 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:46:26.612489 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-eb8be498-d715-4a6c-bce4-cffd4ed2eb14 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:46:26.629717 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-eb8be498-d715-4a6c-bce4-cffd4ed2eb14 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:46:26.633216 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-eb8be498-d715-4a6c-bce4-cffd4ed2eb14 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:46:26.635267 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-eb8be498-d715-4a6c-bce4-cffd4ed2eb14 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] 10.222.0.26 "GET /compute/v2.1/servers/85efeeae-0648-4a33-bc74-c0f0d25880aa" status: 200 len: 1723 microversion: 2.20 time: 0.397747 Oct 19 00:46:26.636287 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 553/668] 10.222.0.26 () {60 vars in 1277 bytes} [Wed Oct 19 00:46:26 2022] GET /compute/v2.1/servers/85efeeae-0648-4a33-bc74-c0f0d25880aa => generated 1723 bytes in 400 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 19 00:46:26.645683 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-bee81930-e0c4-4df2-9a66-6b73614c2cd4 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeTestJSON-server-2056416668", "imageRef": "b387b4f8-f75e-4a68-b49c-a015bf61c46b", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "68404256-b068-4f6b-bb54-21abe9c04e36"}], "security_groups": [{"name": "tempest-securitygroup--1546407613"}], "key_name": "tempest-keypair-1137457954", "user_data": "}} {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 19 00:46:26.862238 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.network.neutron [None req-bee81930-e0c4-4df2-9a66-6b73614c2cd4 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] validate_networks() for [('68404256-b068-4f6b-bb54-21abe9c04e36', None, None, None, None, None)] {{(pid=125641) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 19 00:46:27.342863 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.virt.hardware [None req-bee81930-e0c4-4df2-9a66-6b73614c2cd4 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Flavor limits 0:0:0 {{(pid=125641) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 19 00:46:27.344108 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.virt.hardware [None req-bee81930-e0c4-4df2-9a66-6b73614c2cd4 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Image limits 0:0:0 {{(pid=125641) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 19 00:46:27.345178 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.virt.hardware [None req-bee81930-e0c4-4df2-9a66-6b73614c2cd4 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Flavor pref 0:0:0 {{(pid=125641) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 19 00:46:27.345887 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.virt.hardware [None req-bee81930-e0c4-4df2-9a66-6b73614c2cd4 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Image pref 0:0:0 {{(pid=125641) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 19 00:46:27.346573 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.virt.hardware [None req-bee81930-e0c4-4df2-9a66-6b73614c2cd4 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=125641) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 19 00:46:27.359639 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-bee81930-e0c4-4df2-9a66-6b73614c2cd4 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:46:27.360271 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-bee81930-e0c4-4df2-9a66-6b73614c2cd4 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:46:27.361002 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-bee81930-e0c4-4df2-9a66-6b73614c2cd4 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:46:27.362031 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-bee81930-e0c4-4df2-9a66-6b73614c2cd4 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:46:27.362621 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-bee81930-e0c4-4df2-9a66-6b73614c2cd4 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:46:27.363218 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-bee81930-e0c4-4df2-9a66-6b73614c2cd4 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:46:27.397925 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.quota [None req-bee81930-e0c4-4df2-9a66-6b73614c2cd4 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Getting quotas for project 82d7723ef95b4c3faeac9835eeb233a5. Resources: {'cores', 'ram', 'instances'} {{(pid=125641) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 19 00:46:27.408281 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.quota [None req-bee81930-e0c4-4df2-9a66-6b73614c2cd4 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Getting quotas for user a48ed2d9ded741ca91afc1fbbd9c608f and project 82d7723ef95b4c3faeac9835eeb233a5. Resources: {'cores', 'ram', 'instances'} {{(pid=125641) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 19 00:46:27.433555 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-bee81930-e0c4-4df2-9a66-6b73614c2cd4 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Going to run 1 instances... {{(pid=125641) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 19 00:46:27.438135 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-bee81930-e0c4-4df2-9a66-6b73614c2cd4 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] [instance: b38d4219-8912-4fd9-9d02-7848016f7954] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='b387b4f8-f75e-4a68-b49c-a015bf61c46b',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=125641) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 19 00:46:27.542778 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-bee81930-e0c4-4df2-9a66-6b73614c2cd4 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 202 len: 402 microversion: 2.1 time: 0.903942 Oct 19 00:46:27.544956 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 554/669] 10.222.0.26 () {60 vars in 1147 bytes} [Wed Oct 19 00:46:26 2022] POST /compute/v2.1/servers => generated 402 bytes in 908 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 19 00:46:27.674040 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [req-32139771-0203-4027-9adc-c302b4cb958f req-74715077-2be2-4e2f-a832-d83e35df2e14 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "0da74d88-6672-4df0-b665-b9eec2d073c5", "name": "network-vif-plugged", "status": "completed", "tag": "6ccabad6-2670-419e-95ab-c0c5d0749b3a"}]} {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 19 00:46:27.703239 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-32139771-0203-4027-9adc-c302b4cb958f req-74715077-2be2-4e2f-a832-d83e35df2e14 service nova] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:46:27.704217 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-32139771-0203-4027-9adc-c302b4cb958f req-74715077-2be2-4e2f-a832-d83e35df2e14 service nova] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:46:27.705109 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-32139771-0203-4027-9adc-c302b4cb958f req-74715077-2be2-4e2f-a832-d83e35df2e14 service nova] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:46:27.729321 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.compute.server_external_events [req-32139771-0203-4027-9adc-c302b4cb958f req-74715077-2be2-4e2f-a832-d83e35df2e14 service nova] Creating event network-vif-plugged:6ccabad6-2670-419e-95ab-c0c5d0749b3a for instance 0da74d88-6672-4df0-b665-b9eec2d073c5 on hv-cinder-maste Oct 19 00:46:27.739112 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [req-32139771-0203-4027-9adc-c302b4cb958f req-74715077-2be2-4e2f-a832-d83e35df2e14 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.191372 Oct 19 00:46:27.740003 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 555/670] 10.222.0.26 () {64 vars in 1317 bytes} [Wed Oct 19 00:46:27 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 194 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 19 00:46:27.749008 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-50d2de83-824c-4e1f-8f68-2359ec6af91c tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:46:27.749841 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-50d2de83-824c-4e1f-8f68-2359ec6af91c tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] [instance: 0da74d88-6672-4df0-b665-b9eec2d073c5] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:46:27.757781 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-50d2de83-824c-4e1f-8f68-2359ec6af91c tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:46:27.758266 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-50d2de83-824c-4e1f-8f68-2359ec6af91c tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:46:27.758882 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-50d2de83-824c-4e1f-8f68-2359ec6af91c tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:46:27.759718 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-50d2de83-824c-4e1f-8f68-2359ec6af91c tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:46:27.760259 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-50d2de83-824c-4e1f-8f68-2359ec6af91c tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:46:27.760771 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-50d2de83-824c-4e1f-8f68-2359ec6af91c tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:46:28.028495 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-50d2de83-824c-4e1f-8f68-2359ec6af91c tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f8725c4d05b4dd79b05b43fba15980b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0b35c2038f6b4d0a9012b004f1416c00', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:46:28.044761 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-50d2de83-824c-4e1f-8f68-2359ec6af91c tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] 10.222.0.26 "GET /compute/v2.1/servers/0da74d88-6672-4df0-b665-b9eec2d073c5" status: 200 len: 1367 microversion: 2.1 time: 0.300701 Oct 19 00:46:28.045711 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 556/671] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:46:27 2022] GET /compute/v2.1/servers/0da74d88-6672-4df0-b665-b9eec2d073c5 => generated 1367 bytes in 303 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:46:28.052834 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-689b37ee-da21-46df-9dc3-4df9d7ae0f48 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:46:28.054416 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-689b37ee-da21-46df-9dc3-4df9d7ae0f48 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] [instance: b38d4219-8912-4fd9-9d02-7848016f7954] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:46:28.139636 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-689b37ee-da21-46df-9dc3-4df9d7ae0f48 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a48ed2d9ded741ca91afc1fbbd9c608f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '82d7723ef95b4c3faeac9835eeb233a5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:46:28.148753 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-689b37ee-da21-46df-9dc3-4df9d7ae0f48 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] 10.222.0.26 "GET /compute/v2.1/servers/b38d4219-8912-4fd9-9d02-7848016f7954" status: 200 len: 1238 microversion: 2.1 time: 0.101475 Oct 19 00:46:28.149940 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 557/672] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:46:28 2022] GET /compute/v2.1/servers/b38d4219-8912-4fd9-9d02-7848016f7954 => generated 1238 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:46:28.155503 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-ec6fbec3-b2b0-4324-bf46-8803caee53fe tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:46:28.156232 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-ec6fbec3-b2b0-4324-bf46-8803caee53fe tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] [instance: 85efeeae-0648-4a33-bc74-c0f0d25880aa] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:46:28.158393 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG dbcounter [-] [125641] Writing DB stats nova_api:SELECT=92,nova_api:INSERT=8 {{(pid=125641) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 00:46:28.166652 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ec6fbec3-b2b0-4324-bf46-8803caee53fe tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:46:28.167534 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ec6fbec3-b2b0-4324-bf46-8803caee53fe tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:46:28.167534 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ec6fbec3-b2b0-4324-bf46-8803caee53fe tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:46:28.168239 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ec6fbec3-b2b0-4324-bf46-8803caee53fe tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:46:28.168705 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ec6fbec3-b2b0-4324-bf46-8803caee53fe tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:46:28.169134 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ec6fbec3-b2b0-4324-bf46-8803caee53fe tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:46:28.391846 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-ec6fbec3-b2b0-4324-bf46-8803caee53fe tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:46:28.409048 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-ec6fbec3-b2b0-4324-bf46-8803caee53fe tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:46:28.411658 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-ec6fbec3-b2b0-4324-bf46-8803caee53fe tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:46:28.414470 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-ec6fbec3-b2b0-4324-bf46-8803caee53fe tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] 10.222.0.26 "GET /compute/v2.1/servers/85efeeae-0648-4a33-bc74-c0f0d25880aa" status: 200 len: 1723 microversion: 2.20 time: 0.263013 Oct 19 00:46:28.415504 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 558/673] 10.222.0.26 () {60 vars in 1277 bytes} [Wed Oct 19 00:46:28 2022] GET /compute/v2.1/servers/85efeeae-0648-4a33-bc74-c0f0d25880aa => generated 1723 bytes in 265 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 19 00:46:29.091386 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-92ba160b-359f-4cca-bfb3-d15fadb9c87f tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:46:29.092192 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-92ba160b-359f-4cca-bfb3-d15fadb9c87f tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] [instance: 0da74d88-6672-4df0-b665-b9eec2d073c5] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:46:29.102784 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-92ba160b-359f-4cca-bfb3-d15fadb9c87f tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:46:29.103527 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-92ba160b-359f-4cca-bfb3-d15fadb9c87f tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:46:29.104233 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-92ba160b-359f-4cca-bfb3-d15fadb9c87f tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:46:29.105369 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-92ba160b-359f-4cca-bfb3-d15fadb9c87f tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:46:29.106114 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-92ba160b-359f-4cca-bfb3-d15fadb9c87f tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:46:29.106819 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-92ba160b-359f-4cca-bfb3-d15fadb9c87f tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:46:29.321981 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-92ba160b-359f-4cca-bfb3-d15fadb9c87f tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f8725c4d05b4dd79b05b43fba15980b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0b35c2038f6b4d0a9012b004f1416c00', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:46:29.335821 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-92ba160b-359f-4cca-bfb3-d15fadb9c87f tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] 10.222.0.26 "GET /compute/v2.1/servers/0da74d88-6672-4df0-b665-b9eec2d073c5" status: 200 len: 1367 microversion: 2.1 time: 0.248005 Oct 19 00:46:29.336562 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 559/674] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:46:29 2022] GET /compute/v2.1/servers/0da74d88-6672-4df0-b665-b9eec2d073c5 => generated 1367 bytes in 250 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 19 00:46:29.345025 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-3ccb47df-8bf8-49cb-b119-993b6f1241ad tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:46:29.345989 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-3ccb47df-8bf8-49cb-b119-993b6f1241ad tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] [instance: b38d4219-8912-4fd9-9d02-7848016f7954] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:46:29.357068 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-3ccb47df-8bf8-49cb-b119-993b6f1241ad tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:46:29.357910 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-3ccb47df-8bf8-49cb-b119-993b6f1241ad tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:46:29.358867 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-3ccb47df-8bf8-49cb-b119-993b6f1241ad tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:46:29.360285 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-3ccb47df-8bf8-49cb-b119-993b6f1241ad tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:46:29.360923 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-3ccb47df-8bf8-49cb-b119-993b6f1241ad tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:46:29.362040 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-3ccb47df-8bf8-49cb-b119-993b6f1241ad tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:46:29.488496 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-3ccb47df-8bf8-49cb-b119-993b6f1241ad tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a48ed2d9ded741ca91afc1fbbd9c608f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '82d7723ef95b4c3faeac9835eeb233a5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:46:29.512319 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-3ccb47df-8bf8-49cb-b119-993b6f1241ad tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] 10.222.0.26 "GET /compute/v2.1/servers/b38d4219-8912-4fd9-9d02-7848016f7954" status: 200 len: 1234 microversion: 2.1 time: 0.173431 Oct 19 00:46:29.513344 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 560/675] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:46:29 2022] GET /compute/v2.1/servers/b38d4219-8912-4fd9-9d02-7848016f7954 => generated 1234 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:46:29.518576 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-d78492be-ca93-4d15-8885-7d25a148d995 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:46:29.519491 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-d78492be-ca93-4d15-8885-7d25a148d995 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] [instance: 85efeeae-0648-4a33-bc74-c0f0d25880aa] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:46:29.531524 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d78492be-ca93-4d15-8885-7d25a148d995 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:46:29.532180 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d78492be-ca93-4d15-8885-7d25a148d995 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:46:29.532757 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d78492be-ca93-4d15-8885-7d25a148d995 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:46:29.534058 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d78492be-ca93-4d15-8885-7d25a148d995 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:46:29.534635 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d78492be-ca93-4d15-8885-7d25a148d995 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:46:29.535342 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d78492be-ca93-4d15-8885-7d25a148d995 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:46:29.746858 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-d78492be-ca93-4d15-8885-7d25a148d995 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:46:29.761502 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-d78492be-ca93-4d15-8885-7d25a148d995 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:46:29.763538 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-d78492be-ca93-4d15-8885-7d25a148d995 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:46:29.765344 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-d78492be-ca93-4d15-8885-7d25a148d995 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] 10.222.0.26 "GET /compute/v2.1/servers/85efeeae-0648-4a33-bc74-c0f0d25880aa" status: 200 len: 1744 microversion: 2.20 time: 0.250602 Oct 19 00:46:29.766535 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 561/676] 10.222.0.26 () {60 vars in 1277 bytes} [Wed Oct 19 00:46:29 2022] GET /compute/v2.1/servers/85efeeae-0648-4a33-bc74-c0f0d25880aa => generated 1744 bytes in 252 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 19 00:46:30.381965 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-e1670d11-4e1b-43cf-92d7-b7ab600bfec9 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:46:30.383121 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-e1670d11-4e1b-43cf-92d7-b7ab600bfec9 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] [instance: 0da74d88-6672-4df0-b665-b9eec2d073c5] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:46:30.390523 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-e1670d11-4e1b-43cf-92d7-b7ab600bfec9 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:46:30.391020 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-e1670d11-4e1b-43cf-92d7-b7ab600bfec9 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:46:30.391459 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-e1670d11-4e1b-43cf-92d7-b7ab600bfec9 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:46:30.392180 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-e1670d11-4e1b-43cf-92d7-b7ab600bfec9 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:46:30.392670 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-e1670d11-4e1b-43cf-92d7-b7ab600bfec9 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:46:30.393090 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-e1670d11-4e1b-43cf-92d7-b7ab600bfec9 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:46:30.562330 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-e1670d11-4e1b-43cf-92d7-b7ab600bfec9 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f8725c4d05b4dd79b05b43fba15980b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0b35c2038f6b4d0a9012b004f1416c00', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:46:30.572180 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-e1670d11-4e1b-43cf-92d7-b7ab600bfec9 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] 10.222.0.26 "GET /compute/v2.1/servers/0da74d88-6672-4df0-b665-b9eec2d073c5" status: 200 len: 1555 microversion: 2.1 time: 0.194506 Oct 19 00:46:30.572962 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 562/677] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:46:30 2022] GET /compute/v2.1/servers/0da74d88-6672-4df0-b665-b9eec2d073c5 => generated 1555 bytes in 196 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 19 00:46:30.579415 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-497b5c14-35f3-4958-929c-d4c985797e7e tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:46:30.580207 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-497b5c14-35f3-4958-929c-d4c985797e7e tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] [instance: b38d4219-8912-4fd9-9d02-7848016f7954] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:46:30.588175 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-497b5c14-35f3-4958-929c-d4c985797e7e tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:46:30.588823 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-497b5c14-35f3-4958-929c-d4c985797e7e tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:46:30.589319 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-497b5c14-35f3-4958-929c-d4c985797e7e tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:46:30.590133 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-497b5c14-35f3-4958-929c-d4c985797e7e tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:46:30.590705 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-497b5c14-35f3-4958-929c-d4c985797e7e tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:46:30.591194 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-497b5c14-35f3-4958-929c-d4c985797e7e tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:46:30.687392 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-497b5c14-35f3-4958-929c-d4c985797e7e tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a48ed2d9ded741ca91afc1fbbd9c608f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '82d7723ef95b4c3faeac9835eeb233a5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:46:30.697093 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-497b5c14-35f3-4958-929c-d4c985797e7e tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] 10.222.0.26 "GET /compute/v2.1/servers/b38d4219-8912-4fd9-9d02-7848016f7954" status: 200 len: 1290 microversion: 2.1 time: 0.121850 Oct 19 00:46:30.703013 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 563/678] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:46:30 2022] GET /compute/v2.1/servers/b38d4219-8912-4fd9-9d02-7848016f7954 => generated 1290 bytes in 124 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:46:30.703984 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-c25a9846-d4d1-445c-966f-58d5eaf5c694 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:46:30.704530 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-c25a9846-d4d1-445c-966f-58d5eaf5c694 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] [instance: 0da74d88-6672-4df0-b665-b9eec2d073c5] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:46:30.714223 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-c25a9846-d4d1-445c-966f-58d5eaf5c694 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:46:30.715650 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-c25a9846-d4d1-445c-966f-58d5eaf5c694 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:46:30.716041 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-c25a9846-d4d1-445c-966f-58d5eaf5c694 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:46:30.717340 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-c25a9846-d4d1-445c-966f-58d5eaf5c694 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:46:30.718173 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-c25a9846-d4d1-445c-966f-58d5eaf5c694 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:46:30.718937 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-c25a9846-d4d1-445c-966f-58d5eaf5c694 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:46:30.866090 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-c25a9846-d4d1-445c-966f-58d5eaf5c694 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] 10.222.0.26 "GET /compute/v2.1/servers/0da74d88-6672-4df0-b665-b9eec2d073c5/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.166727 Oct 19 00:46:30.866862 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 564/679] 10.222.0.26 () {58 vars in 1275 bytes} [Wed Oct 19 00:46:30 2022] GET /compute/v2.1/servers/0da74d88-6672-4df0-b665-b9eec2d073c5/os-interface => generated 284 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 19 00:46:30.874003 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-43221e76-94b5-45c5-bacb-8782b2991707 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:46:30.874574 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-43221e76-94b5-45c5-bacb-8782b2991707 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] [instance: 85efeeae-0648-4a33-bc74-c0f0d25880aa] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:46:30.882109 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-43221e76-94b5-45c5-bacb-8782b2991707 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:46:30.882675 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-43221e76-94b5-45c5-bacb-8782b2991707 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:46:30.883125 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-43221e76-94b5-45c5-bacb-8782b2991707 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:46:30.883870 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-43221e76-94b5-45c5-bacb-8782b2991707 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:46:30.884355 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-43221e76-94b5-45c5-bacb-8782b2991707 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:46:30.884774 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-43221e76-94b5-45c5-bacb-8782b2991707 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:46:31.082731 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-43221e76-94b5-45c5-bacb-8782b2991707 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:46:31.091871 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-43221e76-94b5-45c5-bacb-8782b2991707 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:46:31.093330 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-43221e76-94b5-45c5-bacb-8782b2991707 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:46:31.094828 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-43221e76-94b5-45c5-bacb-8782b2991707 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] 10.222.0.26 "GET /compute/v2.1/servers/85efeeae-0648-4a33-bc74-c0f0d25880aa" status: 200 len: 1744 microversion: 2.20 time: 0.226731 Oct 19 00:46:31.095722 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 565/680] 10.222.0.26 () {60 vars in 1277 bytes} [Wed Oct 19 00:46:30 2022] GET /compute/v2.1/servers/85efeeae-0648-4a33-bc74-c0f0d25880aa => generated 1744 bytes in 228 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 19 00:46:31.730233 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-cfa7ca01-64f7-4d40-a72e-2a48d3c242cf tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:46:31.731260 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-cfa7ca01-64f7-4d40-a72e-2a48d3c242cf tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] [instance: b38d4219-8912-4fd9-9d02-7848016f7954] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:46:31.738536 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-cfa7ca01-64f7-4d40-a72e-2a48d3c242cf tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:46:31.739059 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-cfa7ca01-64f7-4d40-a72e-2a48d3c242cf tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:46:31.739525 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-cfa7ca01-64f7-4d40-a72e-2a48d3c242cf tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:46:31.741003 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-cfa7ca01-64f7-4d40-a72e-2a48d3c242cf tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:46:31.741781 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-cfa7ca01-64f7-4d40-a72e-2a48d3c242cf tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:46:31.742803 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-cfa7ca01-64f7-4d40-a72e-2a48d3c242cf tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:46:31.831929 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-cfa7ca01-64f7-4d40-a72e-2a48d3c242cf tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a48ed2d9ded741ca91afc1fbbd9c608f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '82d7723ef95b4c3faeac9835eeb233a5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:46:31.842841 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-cfa7ca01-64f7-4d40-a72e-2a48d3c242cf tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] 10.222.0.26 "GET /compute/v2.1/servers/b38d4219-8912-4fd9-9d02-7848016f7954" status: 200 len: 1296 microversion: 2.1 time: 0.116219 Oct 19 00:46:31.859167 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 566/681] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:46:31 2022] GET /compute/v2.1/servers/b38d4219-8912-4fd9-9d02-7848016f7954 => generated 1296 bytes in 134 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 19 00:46:32.140928 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-c7a417de-f81d-4acf-b213-0f2cbd7ba4a0 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:46:32.142036 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-c7a417de-f81d-4acf-b213-0f2cbd7ba4a0 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] [instance: 85efeeae-0648-4a33-bc74-c0f0d25880aa] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:46:32.152567 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-c7a417de-f81d-4acf-b213-0f2cbd7ba4a0 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:46:32.153450 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-c7a417de-f81d-4acf-b213-0f2cbd7ba4a0 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:46:32.154205 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-c7a417de-f81d-4acf-b213-0f2cbd7ba4a0 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:46:32.155321 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-c7a417de-f81d-4acf-b213-0f2cbd7ba4a0 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:46:32.156062 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-c7a417de-f81d-4acf-b213-0f2cbd7ba4a0 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:46:32.156841 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-c7a417de-f81d-4acf-b213-0f2cbd7ba4a0 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:46:32.415974 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-c7a417de-f81d-4acf-b213-0f2cbd7ba4a0 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:46:32.457616 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-c7a417de-f81d-4acf-b213-0f2cbd7ba4a0 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:46:32.463579 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-c7a417de-f81d-4acf-b213-0f2cbd7ba4a0 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:46:32.468626 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-c7a417de-f81d-4acf-b213-0f2cbd7ba4a0 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] 10.222.0.26 "GET /compute/v2.1/servers/85efeeae-0648-4a33-bc74-c0f0d25880aa" status: 200 len: 1739 microversion: 2.20 time: 0.331517 Oct 19 00:46:32.504833 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 567/682] 10.222.0.26 () {60 vars in 1277 bytes} [Wed Oct 19 00:46:32 2022] GET /compute/v2.1/servers/85efeeae-0648-4a33-bc74-c0f0d25880aa => generated 1739 bytes in 369 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 19 00:46:32.874816 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [req-5d968946-fb84-41ce-8aba-afccef557c6b req-393a9e06-8c04-4431-ad32-131874eea96f service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "85efeeae-0648-4a33-bc74-c0f0d25880aa", "name": "network-vif-plugged", "status": "completed", "tag": "ba933942-134c-497e-8475-ff284eee237e"}]} {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 19 00:46:32.889627 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-5d968946-fb84-41ce-8aba-afccef557c6b req-393a9e06-8c04-4431-ad32-131874eea96f service nova] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:46:32.890751 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-5d968946-fb84-41ce-8aba-afccef557c6b req-393a9e06-8c04-4431-ad32-131874eea96f service nova] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:46:32.891840 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-5d968946-fb84-41ce-8aba-afccef557c6b req-393a9e06-8c04-4431-ad32-131874eea96f service nova] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:46:32.908109 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.compute.server_external_events [req-5d968946-fb84-41ce-8aba-afccef557c6b req-393a9e06-8c04-4431-ad32-131874eea96f service nova] Creating event network-vif-plugged:ba933942-134c-497e-8475-ff284eee237e for instance 85efeeae-0648-4a33-bc74-c0f0d25880aa on hv-cinder-maste Oct 19 00:46:32.917624 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [req-5d968946-fb84-41ce-8aba-afccef557c6b req-393a9e06-8c04-4431-ad32-131874eea96f service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.046634 Oct 19 00:46:32.919059 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 568/683] 10.222.0.26 () {64 vars in 1317 bytes} [Wed Oct 19 00:46:32 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 19 00:46:32.926020 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-52099c28-b295-4c7a-ac87-a6846c03ca42 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:46:32.929316 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-52099c28-b295-4c7a-ac87-a6846c03ca42 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] [instance: b38d4219-8912-4fd9-9d02-7848016f7954] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:46:32.937862 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-52099c28-b295-4c7a-ac87-a6846c03ca42 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:46:32.938813 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-52099c28-b295-4c7a-ac87-a6846c03ca42 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:46:32.939523 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-52099c28-b295-4c7a-ac87-a6846c03ca42 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:46:32.940513 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-52099c28-b295-4c7a-ac87-a6846c03ca42 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:46:32.941972 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-52099c28-b295-4c7a-ac87-a6846c03ca42 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:46:32.942795 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-52099c28-b295-4c7a-ac87-a6846c03ca42 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:46:33.196439 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-52099c28-b295-4c7a-ac87-a6846c03ca42 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a48ed2d9ded741ca91afc1fbbd9c608f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '82d7723ef95b4c3faeac9835eeb233a5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:46:33.218444 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-52099c28-b295-4c7a-ac87-a6846c03ca42 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] 10.222.0.26 "GET /compute/v2.1/servers/b38d4219-8912-4fd9-9d02-7848016f7954" status: 200 len: 1364 microversion: 2.1 time: 0.296943 Oct 19 00:46:33.220996 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 569/684] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:46:32 2022] GET /compute/v2.1/servers/b38d4219-8912-4fd9-9d02-7848016f7954 => generated 1364 bytes in 301 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:46:33.231065 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [req-9ced1e2a-bf5d-49fa-b155-c4fd63c86a12 req-581c95e3-2787-4fac-9107-9b70ea611581 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "0da74d88-6672-4df0-b665-b9eec2d073c5", "tag": "6ccabad6-2670-419e-95ab-c0c5d0749b3a"}]} {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 19 00:46:33.260821 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-9ced1e2a-bf5d-49fa-b155-c4fd63c86a12 req-581c95e3-2787-4fac-9107-9b70ea611581 service nova] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:46:33.262120 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-9ced1e2a-bf5d-49fa-b155-c4fd63c86a12 req-581c95e3-2787-4fac-9107-9b70ea611581 service nova] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:46:33.263424 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-9ced1e2a-bf5d-49fa-b155-c4fd63c86a12 req-581c95e3-2787-4fac-9107-9b70ea611581 service nova] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:46:33.319123 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.compute.server_external_events [req-9ced1e2a-bf5d-49fa-b155-c4fd63c86a12 req-581c95e3-2787-4fac-9107-9b70ea611581 service nova] Creating event network-changed:6ccabad6-2670-419e-95ab-c0c5d0749b3a for instance 0da74d88-6672-4df0-b665-b9eec2d073c5 on hv-cinder-maste Oct 19 00:46:33.330832 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [req-9ced1e2a-bf5d-49fa-b155-c4fd63c86a12 req-581c95e3-2787-4fac-9107-9b70ea611581 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.106709 Oct 19 00:46:33.332534 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 570/685] 10.222.0.26 () {64 vars in 1317 bytes} [Wed Oct 19 00:46:33 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 111 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 19 00:46:33.517906 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-1a69afbc-c109-45a7-b23c-6635db1be31f tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:46:33.519953 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-1a69afbc-c109-45a7-b23c-6635db1be31f tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] [instance: 85efeeae-0648-4a33-bc74-c0f0d25880aa] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:46:33.534424 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-1a69afbc-c109-45a7-b23c-6635db1be31f tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:46:33.536018 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-1a69afbc-c109-45a7-b23c-6635db1be31f tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:46:33.537373 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-1a69afbc-c109-45a7-b23c-6635db1be31f tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:46:33.540135 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-1a69afbc-c109-45a7-b23c-6635db1be31f tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:46:33.544613 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-1a69afbc-c109-45a7-b23c-6635db1be31f tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:46:33.546280 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-1a69afbc-c109-45a7-b23c-6635db1be31f tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:46:33.826847 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-1a69afbc-c109-45a7-b23c-6635db1be31f tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:46:33.847382 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-1a69afbc-c109-45a7-b23c-6635db1be31f tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:46:33.852375 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-1a69afbc-c109-45a7-b23c-6635db1be31f tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:46:33.855094 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-1a69afbc-c109-45a7-b23c-6635db1be31f tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] 10.222.0.26 "GET /compute/v2.1/servers/85efeeae-0648-4a33-bc74-c0f0d25880aa" status: 200 len: 1739 microversion: 2.20 time: 0.346039 Oct 19 00:46:33.856753 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 571/686] 10.222.0.26 () {60 vars in 1277 bytes} [Wed Oct 19 00:46:33 2022] GET /compute/v2.1/servers/85efeeae-0648-4a33-bc74-c0f0d25880aa => generated 1739 bytes in 350 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 19 00:46:34.248956 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-9ac9eaa4-80a8-4cc7-8bd1-f0013c321f3d tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:46:34.251170 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-9ac9eaa4-80a8-4cc7-8bd1-f0013c321f3d tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] [instance: b38d4219-8912-4fd9-9d02-7848016f7954] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:46:34.267635 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-9ac9eaa4-80a8-4cc7-8bd1-f0013c321f3d tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:46:34.269020 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-9ac9eaa4-80a8-4cc7-8bd1-f0013c321f3d tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:46:34.270352 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-9ac9eaa4-80a8-4cc7-8bd1-f0013c321f3d tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:46:34.273501 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-9ac9eaa4-80a8-4cc7-8bd1-f0013c321f3d tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:46:34.274911 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-9ac9eaa4-80a8-4cc7-8bd1-f0013c321f3d tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:46:34.276019 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-9ac9eaa4-80a8-4cc7-8bd1-f0013c321f3d tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:46:34.627407 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-9ac9eaa4-80a8-4cc7-8bd1-f0013c321f3d tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a48ed2d9ded741ca91afc1fbbd9c608f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '82d7723ef95b4c3faeac9835eeb233a5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:46:34.639098 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG dbcounter [-] [125641] Writing DB stats nova_cell1:SELECT=100 {{(pid=125641) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 00:46:34.652129 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-9ac9eaa4-80a8-4cc7-8bd1-f0013c321f3d tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] 10.222.0.26 "GET /compute/v2.1/servers/b38d4219-8912-4fd9-9d02-7848016f7954" status: 200 len: 1364 microversion: 2.1 time: 0.407250 Oct 19 00:46:34.666626 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 572/687] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:46:34 2022] GET /compute/v2.1/servers/b38d4219-8912-4fd9-9d02-7848016f7954 => generated 1364 bytes in 410 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:46:34.923442 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-ffab38c0-e07c-409a-83d1-7af12c662072 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:46:34.929610 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-ffab38c0-e07c-409a-83d1-7af12c662072 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] [instance: 85efeeae-0648-4a33-bc74-c0f0d25880aa] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:46:34.957166 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ffab38c0-e07c-409a-83d1-7af12c662072 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:46:34.957831 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ffab38c0-e07c-409a-83d1-7af12c662072 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:46:34.958664 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ffab38c0-e07c-409a-83d1-7af12c662072 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:46:34.960023 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ffab38c0-e07c-409a-83d1-7af12c662072 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:46:34.961211 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ffab38c0-e07c-409a-83d1-7af12c662072 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:46:34.962018 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ffab38c0-e07c-409a-83d1-7af12c662072 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:46:35.363884 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-ffab38c0-e07c-409a-83d1-7af12c662072 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:46:35.421914 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-ffab38c0-e07c-409a-83d1-7af12c662072 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:46:35.427870 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-ffab38c0-e07c-409a-83d1-7af12c662072 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:46:35.430839 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-ffab38c0-e07c-409a-83d1-7af12c662072 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] 10.222.0.26 "GET /compute/v2.1/servers/85efeeae-0648-4a33-bc74-c0f0d25880aa" status: 200 len: 1739 microversion: 2.20 time: 0.515203 Oct 19 00:46:35.431983 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 573/688] 10.222.0.26 () {60 vars in 1277 bytes} [Wed Oct 19 00:46:34 2022] GET /compute/v2.1/servers/85efeeae-0648-4a33-bc74-c0f0d25880aa => generated 1739 bytes in 518 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 19 00:46:35.445495 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [req-45e85277-9957-4eab-891e-2f3ad9f29522 req-c481894c-f06b-448b-bc98-0a7f6db61861 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "b38d4219-8912-4fd9-9d02-7848016f7954", "tag": "2b936cf8-a250-46ff-af9b-c10ccce59df7"}]} {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 19 00:46:35.485077 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-45e85277-9957-4eab-891e-2f3ad9f29522 req-c481894c-f06b-448b-bc98-0a7f6db61861 service nova] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:46:35.486092 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-45e85277-9957-4eab-891e-2f3ad9f29522 req-c481894c-f06b-448b-bc98-0a7f6db61861 service nova] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:46:35.487009 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-45e85277-9957-4eab-891e-2f3ad9f29522 req-c481894c-f06b-448b-bc98-0a7f6db61861 service nova] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:46:35.515589 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.compute.server_external_events [req-45e85277-9957-4eab-891e-2f3ad9f29522 req-c481894c-f06b-448b-bc98-0a7f6db61861 service nova] Creating event network-changed:2b936cf8-a250-46ff-af9b-c10ccce59df7 for instance b38d4219-8912-4fd9-9d02-7848016f7954 on hv-cinder-maste Oct 19 00:46:35.526823 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [req-45e85277-9957-4eab-891e-2f3ad9f29522 req-c481894c-f06b-448b-bc98-0a7f6db61861 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.091347 Oct 19 00:46:35.528167 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 574/689] 10.222.0.26 () {64 vars in 1317 bytes} [Wed Oct 19 00:46:35 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 19 00:46:35.687931 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-3f1e5eb3-47a5-4e6e-826c-b0116e3f232e tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:46:35.690455 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-3f1e5eb3-47a5-4e6e-826c-b0116e3f232e tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] [instance: b38d4219-8912-4fd9-9d02-7848016f7954] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:46:35.702379 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-3f1e5eb3-47a5-4e6e-826c-b0116e3f232e tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:46:35.703297 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-3f1e5eb3-47a5-4e6e-826c-b0116e3f232e tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:46:35.703898 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-3f1e5eb3-47a5-4e6e-826c-b0116e3f232e tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:46:35.705323 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-3f1e5eb3-47a5-4e6e-826c-b0116e3f232e tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:46:35.706146 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-3f1e5eb3-47a5-4e6e-826c-b0116e3f232e tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:46:35.707236 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-3f1e5eb3-47a5-4e6e-826c-b0116e3f232e tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:46:36.012502 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-3f1e5eb3-47a5-4e6e-826c-b0116e3f232e tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a48ed2d9ded741ca91afc1fbbd9c608f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '82d7723ef95b4c3faeac9835eeb233a5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:46:36.030871 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-3f1e5eb3-47a5-4e6e-826c-b0116e3f232e tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] 10.222.0.26 "GET /compute/v2.1/servers/b38d4219-8912-4fd9-9d02-7848016f7954" status: 200 len: 1364 microversion: 2.1 time: 0.350560 Oct 19 00:46:36.032941 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 575/690] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:46:35 2022] GET /compute/v2.1/servers/b38d4219-8912-4fd9-9d02-7848016f7954 => generated 1364 bytes in 355 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:46:36.464576 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-7e478bbd-5867-42d9-a50b-10f041f550a3 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:46:36.465607 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-7e478bbd-5867-42d9-a50b-10f041f550a3 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] [instance: 85efeeae-0648-4a33-bc74-c0f0d25880aa] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:46:36.478789 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-7e478bbd-5867-42d9-a50b-10f041f550a3 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:46:36.479665 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-7e478bbd-5867-42d9-a50b-10f041f550a3 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:46:36.480438 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-7e478bbd-5867-42d9-a50b-10f041f550a3 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:46:36.481506 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-7e478bbd-5867-42d9-a50b-10f041f550a3 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:46:36.482130 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-7e478bbd-5867-42d9-a50b-10f041f550a3 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:46:36.482846 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-7e478bbd-5867-42d9-a50b-10f041f550a3 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:46:36.713696 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-7e478bbd-5867-42d9-a50b-10f041f550a3 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:46:36.729464 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-7e478bbd-5867-42d9-a50b-10f041f550a3 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:46:36.731785 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-7e478bbd-5867-42d9-a50b-10f041f550a3 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:46:36.733937 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-7e478bbd-5867-42d9-a50b-10f041f550a3 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] 10.222.0.26 "GET /compute/v2.1/servers/85efeeae-0648-4a33-bc74-c0f0d25880aa" status: 200 len: 1739 microversion: 2.20 time: 0.276687 Oct 19 00:46:36.734968 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 576/691] 10.222.0.26 () {60 vars in 1277 bytes} [Wed Oct 19 00:46:36 2022] GET /compute/v2.1/servers/85efeeae-0648-4a33-bc74-c0f0d25880aa => generated 1739 bytes in 279 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 19 00:46:37.064722 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-f29724f4-b45e-411d-ad4a-ff70f1afee75 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:46:37.067063 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-f29724f4-b45e-411d-ad4a-ff70f1afee75 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] [instance: b38d4219-8912-4fd9-9d02-7848016f7954] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:46:37.093517 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-f29724f4-b45e-411d-ad4a-ff70f1afee75 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:46:37.094315 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-f29724f4-b45e-411d-ad4a-ff70f1afee75 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:46:37.105519 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-f29724f4-b45e-411d-ad4a-ff70f1afee75 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:46:37.116486 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-f29724f4-b45e-411d-ad4a-ff70f1afee75 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:46:37.118386 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-f29724f4-b45e-411d-ad4a-ff70f1afee75 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.010s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:46:37.118386 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-f29724f4-b45e-411d-ad4a-ff70f1afee75 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:46:37.380153 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG dbcounter [-] [125641] Writing DB stats nova_cell0:SELECT=2 {{(pid=125641) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 00:46:37.421326 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-f29724f4-b45e-411d-ad4a-ff70f1afee75 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a48ed2d9ded741ca91afc1fbbd9c608f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '82d7723ef95b4c3faeac9835eeb233a5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:46:37.433146 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-f29724f4-b45e-411d-ad4a-ff70f1afee75 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] 10.222.0.26 "GET /compute/v2.1/servers/b38d4219-8912-4fd9-9d02-7848016f7954" status: 200 len: 1364 microversion: 2.1 time: 0.378176 Oct 19 00:46:37.434016 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 577/692] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:46:37 2022] GET /compute/v2.1/servers/b38d4219-8912-4fd9-9d02-7848016f7954 => generated 1364 bytes in 380 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 19 00:46:37.763482 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-dd9ccc73-cdb0-451f-ad61-66d7b858d243 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:46:37.764620 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-dd9ccc73-cdb0-451f-ad61-66d7b858d243 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] [instance: 85efeeae-0648-4a33-bc74-c0f0d25880aa] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:46:37.773577 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-dd9ccc73-cdb0-451f-ad61-66d7b858d243 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:46:37.777123 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-dd9ccc73-cdb0-451f-ad61-66d7b858d243 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:46:37.779295 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-dd9ccc73-cdb0-451f-ad61-66d7b858d243 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:46:37.780198 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-dd9ccc73-cdb0-451f-ad61-66d7b858d243 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:46:37.780807 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-dd9ccc73-cdb0-451f-ad61-66d7b858d243 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:46:37.781352 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-dd9ccc73-cdb0-451f-ad61-66d7b858d243 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:46:37.990966 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-dd9ccc73-cdb0-451f-ad61-66d7b858d243 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:46:38.007299 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-dd9ccc73-cdb0-451f-ad61-66d7b858d243 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:46:38.010815 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-dd9ccc73-cdb0-451f-ad61-66d7b858d243 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:46:38.030515 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-dd9ccc73-cdb0-451f-ad61-66d7b858d243 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] 10.222.0.26 "GET /compute/v2.1/servers/85efeeae-0648-4a33-bc74-c0f0d25880aa" status: 200 len: 1739 microversion: 2.20 time: 0.253787 Oct 19 00:46:38.030515 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 578/693] 10.222.0.26 () {60 vars in 1277 bytes} [Wed Oct 19 00:46:37 2022] GET /compute/v2.1/servers/85efeeae-0648-4a33-bc74-c0f0d25880aa => generated 1739 bytes in 259 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 19 00:46:38.473627 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-bf8b6082-8903-4189-b6cb-d0aadcf0cd35 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:46:38.475665 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-bf8b6082-8903-4189-b6cb-d0aadcf0cd35 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] [instance: b38d4219-8912-4fd9-9d02-7848016f7954] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:46:38.487029 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG dbcounter [-] [125641] Writing DB stats nova_cell0:SELECT=24 {{(pid=125641) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 00:46:38.511524 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-bf8b6082-8903-4189-b6cb-d0aadcf0cd35 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:46:38.512283 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-bf8b6082-8903-4189-b6cb-d0aadcf0cd35 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:46:38.513707 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-bf8b6082-8903-4189-b6cb-d0aadcf0cd35 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:46:38.514956 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-bf8b6082-8903-4189-b6cb-d0aadcf0cd35 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:46:38.515448 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-bf8b6082-8903-4189-b6cb-d0aadcf0cd35 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:46:38.516026 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-bf8b6082-8903-4189-b6cb-d0aadcf0cd35 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:46:38.689930 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-bf8b6082-8903-4189-b6cb-d0aadcf0cd35 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a48ed2d9ded741ca91afc1fbbd9c608f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '82d7723ef95b4c3faeac9835eeb233a5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:46:38.700180 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-bf8b6082-8903-4189-b6cb-d0aadcf0cd35 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] 10.222.0.26 "GET /compute/v2.1/servers/b38d4219-8912-4fd9-9d02-7848016f7954" status: 200 len: 1364 microversion: 2.1 time: 0.234476 Oct 19 00:46:38.716582 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 579/694] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:46:38 2022] GET /compute/v2.1/servers/b38d4219-8912-4fd9-9d02-7848016f7954 => generated 1364 bytes in 252 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:46:39.074725 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-74548302-1cf1-42a9-8511-8c4aa489c3e4 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:46:39.076865 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-74548302-1cf1-42a9-8511-8c4aa489c3e4 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] [instance: 85efeeae-0648-4a33-bc74-c0f0d25880aa] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:46:39.088535 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-74548302-1cf1-42a9-8511-8c4aa489c3e4 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:46:39.089243 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-74548302-1cf1-42a9-8511-8c4aa489c3e4 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:46:39.090238 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-74548302-1cf1-42a9-8511-8c4aa489c3e4 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:46:39.091720 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-74548302-1cf1-42a9-8511-8c4aa489c3e4 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:46:39.092770 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-74548302-1cf1-42a9-8511-8c4aa489c3e4 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:46:39.093396 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-74548302-1cf1-42a9-8511-8c4aa489c3e4 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:46:39.304621 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-74548302-1cf1-42a9-8511-8c4aa489c3e4 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:46:39.314184 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-74548302-1cf1-42a9-8511-8c4aa489c3e4 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:46:39.317537 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-74548302-1cf1-42a9-8511-8c4aa489c3e4 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:46:39.325167 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-74548302-1cf1-42a9-8511-8c4aa489c3e4 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] 10.222.0.26 "GET /compute/v2.1/servers/85efeeae-0648-4a33-bc74-c0f0d25880aa" status: 200 len: 1739 microversion: 2.20 time: 0.256458 Oct 19 00:46:39.339291 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 580/695] 10.222.0.26 () {60 vars in 1277 bytes} [Wed Oct 19 00:46:39 2022] GET /compute/v2.1/servers/85efeeae-0648-4a33-bc74-c0f0d25880aa => generated 1739 bytes in 273 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 19 00:46:39.724743 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-8d1a9cf2-4237-4f91-bf0a-03785ccb021a tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:46:39.725372 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-8d1a9cf2-4237-4f91-bf0a-03785ccb021a tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] [instance: b38d4219-8912-4fd9-9d02-7848016f7954] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:46:39.733814 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-8d1a9cf2-4237-4f91-bf0a-03785ccb021a tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:46:39.734307 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-8d1a9cf2-4237-4f91-bf0a-03785ccb021a tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:46:39.734777 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-8d1a9cf2-4237-4f91-bf0a-03785ccb021a tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:46:39.735578 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-8d1a9cf2-4237-4f91-bf0a-03785ccb021a tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:46:39.736080 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-8d1a9cf2-4237-4f91-bf0a-03785ccb021a tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:46:39.736521 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-8d1a9cf2-4237-4f91-bf0a-03785ccb021a tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:46:39.993908 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-8d1a9cf2-4237-4f91-bf0a-03785ccb021a tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a48ed2d9ded741ca91afc1fbbd9c608f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '82d7723ef95b4c3faeac9835eeb233a5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:46:40.012440 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-8d1a9cf2-4237-4f91-bf0a-03785ccb021a tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] 10.222.0.26 "GET /compute/v2.1/servers/b38d4219-8912-4fd9-9d02-7848016f7954" status: 200 len: 1364 microversion: 2.1 time: 0.289104 Oct 19 00:46:40.012440 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 581/696] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:46:39 2022] GET /compute/v2.1/servers/b38d4219-8912-4fd9-9d02-7848016f7954 => generated 1364 bytes in 292 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 19 00:46:40.385873 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-f06000e9-ee18-44df-a391-ebf206f56019 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:46:40.388149 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-f06000e9-ee18-44df-a391-ebf206f56019 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] [instance: 85efeeae-0648-4a33-bc74-c0f0d25880aa] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:46:40.398187 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-f06000e9-ee18-44df-a391-ebf206f56019 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:46:40.398918 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-f06000e9-ee18-44df-a391-ebf206f56019 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:46:40.399863 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-f06000e9-ee18-44df-a391-ebf206f56019 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:46:40.401325 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-f06000e9-ee18-44df-a391-ebf206f56019 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:46:40.402623 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-f06000e9-ee18-44df-a391-ebf206f56019 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:46:40.403591 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-f06000e9-ee18-44df-a391-ebf206f56019 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:46:40.630943 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-f06000e9-ee18-44df-a391-ebf206f56019 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:46:40.657352 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-f06000e9-ee18-44df-a391-ebf206f56019 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:46:40.657352 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-f06000e9-ee18-44df-a391-ebf206f56019 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:46:40.657352 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-f06000e9-ee18-44df-a391-ebf206f56019 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] 10.222.0.26 "GET /compute/v2.1/servers/85efeeae-0648-4a33-bc74-c0f0d25880aa" status: 200 len: 1739 microversion: 2.20 time: 0.271960 Oct 19 00:46:40.658056 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 582/697] 10.222.0.26 () {60 vars in 1277 bytes} [Wed Oct 19 00:46:40 2022] GET /compute/v2.1/servers/85efeeae-0648-4a33-bc74-c0f0d25880aa => generated 1739 bytes in 274 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 19 00:46:40.986246 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [req-8639699e-38a2-4712-a674-141e1bc6cd87 req-9d7c3949-5d0f-4569-904e-1d082c6128be service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "b38d4219-8912-4fd9-9d02-7848016f7954", "name": "network-vif-plugged", "status": "completed", "tag": "2b936cf8-a250-46ff-af9b-c10ccce59df7"}]} {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 19 00:46:41.003449 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-8639699e-38a2-4712-a674-141e1bc6cd87 req-9d7c3949-5d0f-4569-904e-1d082c6128be service nova] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:46:41.008598 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-8639699e-38a2-4712-a674-141e1bc6cd87 req-9d7c3949-5d0f-4569-904e-1d082c6128be service nova] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:46:41.009376 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-8639699e-38a2-4712-a674-141e1bc6cd87 req-9d7c3949-5d0f-4569-904e-1d082c6128be service nova] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:46:41.027381 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.compute.server_external_events [req-8639699e-38a2-4712-a674-141e1bc6cd87 req-9d7c3949-5d0f-4569-904e-1d082c6128be service nova] Creating event network-vif-plugged:2b936cf8-a250-46ff-af9b-c10ccce59df7 for instance b38d4219-8912-4fd9-9d02-7848016f7954 on hv-cinder-maste Oct 19 00:46:41.032880 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [req-8639699e-38a2-4712-a674-141e1bc6cd87 req-9d7c3949-5d0f-4569-904e-1d082c6128be service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.050462 Oct 19 00:46:41.033913 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 583/698] 10.222.0.26 () {64 vars in 1317 bytes} [Wed Oct 19 00:46:40 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 19 00:46:41.058534 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-3ce7c051-3a58-4093-a77b-6045477ebea1 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:46:41.059781 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-3ce7c051-3a58-4093-a77b-6045477ebea1 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] [instance: b38d4219-8912-4fd9-9d02-7848016f7954] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:46:41.067424 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-3ce7c051-3a58-4093-a77b-6045477ebea1 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:46:41.068942 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-3ce7c051-3a58-4093-a77b-6045477ebea1 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:46:41.069559 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-3ce7c051-3a58-4093-a77b-6045477ebea1 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:46:41.071071 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-3ce7c051-3a58-4093-a77b-6045477ebea1 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:46:41.071071 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-3ce7c051-3a58-4093-a77b-6045477ebea1 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:46:41.071616 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-3ce7c051-3a58-4093-a77b-6045477ebea1 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:46:41.305163 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-3ce7c051-3a58-4093-a77b-6045477ebea1 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a48ed2d9ded741ca91afc1fbbd9c608f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '82d7723ef95b4c3faeac9835eeb233a5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:46:41.313747 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-3ce7c051-3a58-4093-a77b-6045477ebea1 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] 10.222.0.26 "GET /compute/v2.1/servers/b38d4219-8912-4fd9-9d02-7848016f7954" status: 200 len: 1364 microversion: 2.1 time: 0.263659 Oct 19 00:46:41.314689 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 584/699] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:46:41 2022] GET /compute/v2.1/servers/b38d4219-8912-4fd9-9d02-7848016f7954 => generated 1364 bytes in 266 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:46:41.703131 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-faeefd8d-cf94-4bea-bc60-72d3b29ba79f tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:46:41.703956 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-faeefd8d-cf94-4bea-bc60-72d3b29ba79f tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] [instance: 85efeeae-0648-4a33-bc74-c0f0d25880aa] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:46:41.711392 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-faeefd8d-cf94-4bea-bc60-72d3b29ba79f tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:46:41.712438 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-faeefd8d-cf94-4bea-bc60-72d3b29ba79f tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:46:41.712983 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-faeefd8d-cf94-4bea-bc60-72d3b29ba79f tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:46:41.714416 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-faeefd8d-cf94-4bea-bc60-72d3b29ba79f tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:46:41.715611 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-faeefd8d-cf94-4bea-bc60-72d3b29ba79f tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:46:41.716107 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-faeefd8d-cf94-4bea-bc60-72d3b29ba79f tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:46:41.951602 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-faeefd8d-cf94-4bea-bc60-72d3b29ba79f tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:46:41.974046 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-faeefd8d-cf94-4bea-bc60-72d3b29ba79f tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:46:41.978035 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-faeefd8d-cf94-4bea-bc60-72d3b29ba79f tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:46:41.980337 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-faeefd8d-cf94-4bea-bc60-72d3b29ba79f tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] 10.222.0.26 "GET /compute/v2.1/servers/85efeeae-0648-4a33-bc74-c0f0d25880aa" status: 200 len: 1739 microversion: 2.20 time: 0.281429 Oct 19 00:46:41.994514 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 585/700] 10.222.0.26 () {60 vars in 1277 bytes} [Wed Oct 19 00:46:41 2022] GET /compute/v2.1/servers/85efeeae-0648-4a33-bc74-c0f0d25880aa => generated 1739 bytes in 284 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 19 00:46:42.362644 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-53be60d2-6f87-4731-b9d7-1ccdd2de8a1f tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:46:42.363808 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-53be60d2-6f87-4731-b9d7-1ccdd2de8a1f tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] [instance: b38d4219-8912-4fd9-9d02-7848016f7954] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:46:42.374915 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-53be60d2-6f87-4731-b9d7-1ccdd2de8a1f tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:46:42.376499 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-53be60d2-6f87-4731-b9d7-1ccdd2de8a1f tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:46:42.379262 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-53be60d2-6f87-4731-b9d7-1ccdd2de8a1f tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:46:42.381100 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-53be60d2-6f87-4731-b9d7-1ccdd2de8a1f tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:46:42.381859 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-53be60d2-6f87-4731-b9d7-1ccdd2de8a1f tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:46:42.382623 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-53be60d2-6f87-4731-b9d7-1ccdd2de8a1f tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:46:42.575822 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-53be60d2-6f87-4731-b9d7-1ccdd2de8a1f tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a48ed2d9ded741ca91afc1fbbd9c608f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '82d7723ef95b4c3faeac9835eeb233a5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:46:42.589618 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-53be60d2-6f87-4731-b9d7-1ccdd2de8a1f tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] 10.222.0.26 "GET /compute/v2.1/servers/b38d4219-8912-4fd9-9d02-7848016f7954" status: 200 len: 1364 microversion: 2.1 time: 0.236666 Oct 19 00:46:42.590718 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 586/701] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:46:42 2022] GET /compute/v2.1/servers/b38d4219-8912-4fd9-9d02-7848016f7954 => generated 1364 bytes in 241 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 19 00:46:43.010297 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-07ec3148-eb1d-4293-9068-fccbad139ee6 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:46:43.011444 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-07ec3148-eb1d-4293-9068-fccbad139ee6 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] [instance: 85efeeae-0648-4a33-bc74-c0f0d25880aa] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:46:43.026970 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-07ec3148-eb1d-4293-9068-fccbad139ee6 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:46:43.028023 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-07ec3148-eb1d-4293-9068-fccbad139ee6 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:46:43.028772 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-07ec3148-eb1d-4293-9068-fccbad139ee6 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:46:43.029985 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-07ec3148-eb1d-4293-9068-fccbad139ee6 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:46:43.033332 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-07ec3148-eb1d-4293-9068-fccbad139ee6 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:46:43.033332 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-07ec3148-eb1d-4293-9068-fccbad139ee6 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:46:43.290838 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-07ec3148-eb1d-4293-9068-fccbad139ee6 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:46:43.312846 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-07ec3148-eb1d-4293-9068-fccbad139ee6 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:46:43.314393 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-07ec3148-eb1d-4293-9068-fccbad139ee6 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:46:43.315718 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-07ec3148-eb1d-4293-9068-fccbad139ee6 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] 10.222.0.26 "GET /compute/v2.1/servers/85efeeae-0648-4a33-bc74-c0f0d25880aa" status: 200 len: 1739 microversion: 2.20 time: 0.312967 Oct 19 00:46:43.316674 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 587/702] 10.222.0.26 () {60 vars in 1277 bytes} [Wed Oct 19 00:46:43 2022] GET /compute/v2.1/servers/85efeeae-0648-4a33-bc74-c0f0d25880aa => generated 1739 bytes in 315 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 19 00:46:43.617140 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-a4de1e7c-f182-48a2-936b-88d38e5f7c45 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:46:43.618333 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-a4de1e7c-f182-48a2-936b-88d38e5f7c45 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] [instance: b38d4219-8912-4fd9-9d02-7848016f7954] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:46:43.628716 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-a4de1e7c-f182-48a2-936b-88d38e5f7c45 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:46:43.629668 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-a4de1e7c-f182-48a2-936b-88d38e5f7c45 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:46:43.631713 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-a4de1e7c-f182-48a2-936b-88d38e5f7c45 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:46:43.633073 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-a4de1e7c-f182-48a2-936b-88d38e5f7c45 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:46:43.634102 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-a4de1e7c-f182-48a2-936b-88d38e5f7c45 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:46:43.635483 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-a4de1e7c-f182-48a2-936b-88d38e5f7c45 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:46:43.892820 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-a4de1e7c-f182-48a2-936b-88d38e5f7c45 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a48ed2d9ded741ca91afc1fbbd9c608f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '82d7723ef95b4c3faeac9835eeb233a5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:46:43.910045 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-a4de1e7c-f182-48a2-936b-88d38e5f7c45 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] 10.222.0.26 "GET /compute/v2.1/servers/b38d4219-8912-4fd9-9d02-7848016f7954" status: 200 len: 1550 microversion: 2.1 time: 0.298645 Oct 19 00:46:43.913767 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 588/703] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:46:43 2022] GET /compute/v2.1/servers/b38d4219-8912-4fd9-9d02-7848016f7954 => generated 1550 bytes in 302 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 19 00:46:43.935767 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-61f1a839-9825-450e-86cc-c7eecd4d510a tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:46:43.936267 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-61f1a839-9825-450e-86cc-c7eecd4d510a tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] [instance: b38d4219-8912-4fd9-9d02-7848016f7954] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:46:43.943890 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-61f1a839-9825-450e-86cc-c7eecd4d510a tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:46:43.944351 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-61f1a839-9825-450e-86cc-c7eecd4d510a tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:46:43.944842 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-61f1a839-9825-450e-86cc-c7eecd4d510a tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:46:43.945571 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-61f1a839-9825-450e-86cc-c7eecd4d510a tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:46:43.946026 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-61f1a839-9825-450e-86cc-c7eecd4d510a tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:46:43.946447 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-61f1a839-9825-450e-86cc-c7eecd4d510a tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:46:44.072083 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-61f1a839-9825-450e-86cc-c7eecd4d510a tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] 10.222.0.26 "GET /compute/v2.1/servers/b38d4219-8912-4fd9-9d02-7848016f7954/os-interface" status: 200 len: 285 microversion: 2.1 time: 0.140878 Oct 19 00:46:44.073520 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 589/704] 10.222.0.26 () {58 vars in 1275 bytes} [Wed Oct 19 00:46:43 2022] GET /compute/v2.1/servers/b38d4219-8912-4fd9-9d02-7848016f7954/os-interface => generated 285 bytes in 143 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 19 00:46:44.383361 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-538ddddb-9538-4cc5-bad3-6ed5f0385965 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:46:44.384674 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-538ddddb-9538-4cc5-bad3-6ed5f0385965 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] [instance: 85efeeae-0648-4a33-bc74-c0f0d25880aa] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:46:44.413019 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-538ddddb-9538-4cc5-bad3-6ed5f0385965 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:46:44.413736 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-538ddddb-9538-4cc5-bad3-6ed5f0385965 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:46:44.414589 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-538ddddb-9538-4cc5-bad3-6ed5f0385965 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:46:44.415874 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-538ddddb-9538-4cc5-bad3-6ed5f0385965 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:46:44.416827 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-538ddddb-9538-4cc5-bad3-6ed5f0385965 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:46:44.418379 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-538ddddb-9538-4cc5-bad3-6ed5f0385965 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:46:44.881601 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-538ddddb-9538-4cc5-bad3-6ed5f0385965 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:46:44.900745 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-538ddddb-9538-4cc5-bad3-6ed5f0385965 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:46:44.904159 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-538ddddb-9538-4cc5-bad3-6ed5f0385965 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:46:44.905916 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-538ddddb-9538-4cc5-bad3-6ed5f0385965 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] 10.222.0.26 "GET /compute/v2.1/servers/85efeeae-0648-4a33-bc74-c0f0d25880aa" status: 200 len: 1739 microversion: 2.20 time: 0.543739 Oct 19 00:46:44.927388 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 590/705] 10.222.0.26 () {60 vars in 1277 bytes} [Wed Oct 19 00:46:44 2022] GET /compute/v2.1/servers/85efeeae-0648-4a33-bc74-c0f0d25880aa => generated 1739 bytes in 566 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 19 00:46:45.971914 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [req-568e52cd-e5c0-4ec6-9a12-075f696104bf req-a87a17e9-020d-472b-9692-ce8ecb2486bf service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "b38d4219-8912-4fd9-9d02-7848016f7954", "tag": "2b936cf8-a250-46ff-af9b-c10ccce59df7"}]} {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 19 00:46:45.987574 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-568e52cd-e5c0-4ec6-9a12-075f696104bf req-a87a17e9-020d-472b-9692-ce8ecb2486bf service nova] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:46:45.988169 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-568e52cd-e5c0-4ec6-9a12-075f696104bf req-a87a17e9-020d-472b-9692-ce8ecb2486bf service nova] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:46:45.988676 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-568e52cd-e5c0-4ec6-9a12-075f696104bf req-a87a17e9-020d-472b-9692-ce8ecb2486bf service nova] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:46:45.994175 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG dbcounter [-] [125641] Writing DB stats nova_cell1:SELECT=100 {{(pid=125641) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 00:46:46.003925 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.compute.server_external_events [req-568e52cd-e5c0-4ec6-9a12-075f696104bf req-a87a17e9-020d-472b-9692-ce8ecb2486bf service nova] Creating event network-changed:2b936cf8-a250-46ff-af9b-c10ccce59df7 for instance b38d4219-8912-4fd9-9d02-7848016f7954 on hv-cinder-maste Oct 19 00:46:46.010299 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [req-568e52cd-e5c0-4ec6-9a12-075f696104bf req-a87a17e9-020d-472b-9692-ce8ecb2486bf service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.124645 Oct 19 00:46:46.011052 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 591/706] 10.222.0.26 () {64 vars in 1317 bytes} [Wed Oct 19 00:46:45 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 127 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 19 00:46:46.017870 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-3d54c6a9-cd7a-4bc2-b699-cf69f5024434 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:46:46.018746 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-3d54c6a9-cd7a-4bc2-b699-cf69f5024434 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] [instance: 85efeeae-0648-4a33-bc74-c0f0d25880aa] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:46:46.026664 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-3d54c6a9-cd7a-4bc2-b699-cf69f5024434 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:46:46.027259 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-3d54c6a9-cd7a-4bc2-b699-cf69f5024434 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:46:46.027763 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-3d54c6a9-cd7a-4bc2-b699-cf69f5024434 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:46:46.028837 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-3d54c6a9-cd7a-4bc2-b699-cf69f5024434 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:46:46.029518 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-3d54c6a9-cd7a-4bc2-b699-cf69f5024434 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:46:46.030026 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-3d54c6a9-cd7a-4bc2-b699-cf69f5024434 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:46:46.250520 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-3d54c6a9-cd7a-4bc2-b699-cf69f5024434 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:46:46.261924 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-3d54c6a9-cd7a-4bc2-b699-cf69f5024434 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:46:46.263676 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-3d54c6a9-cd7a-4bc2-b699-cf69f5024434 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:46:46.265117 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-3d54c6a9-cd7a-4bc2-b699-cf69f5024434 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] 10.222.0.26 "GET /compute/v2.1/servers/85efeeae-0648-4a33-bc74-c0f0d25880aa" status: 200 len: 1721 microversion: 2.20 time: 0.252649 Oct 19 00:46:46.266100 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 592/707] 10.222.0.26 () {60 vars in 1277 bytes} [Wed Oct 19 00:46:46 2022] GET /compute/v2.1/servers/85efeeae-0648-4a33-bc74-c0f0d25880aa => generated 1721 bytes in 254 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 19 00:46:47.297244 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-184f499e-e354-4275-8ca2-f4cba48b8416 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:46:47.298302 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-184f499e-e354-4275-8ca2-f4cba48b8416 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] [instance: 85efeeae-0648-4a33-bc74-c0f0d25880aa] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:46:47.307782 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-184f499e-e354-4275-8ca2-f4cba48b8416 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:46:47.308428 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-184f499e-e354-4275-8ca2-f4cba48b8416 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:46:47.309044 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-184f499e-e354-4275-8ca2-f4cba48b8416 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:46:47.310075 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-184f499e-e354-4275-8ca2-f4cba48b8416 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:46:47.310751 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-184f499e-e354-4275-8ca2-f4cba48b8416 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:46:47.311430 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-184f499e-e354-4275-8ca2-f4cba48b8416 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:46:47.477764 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-184f499e-e354-4275-8ca2-f4cba48b8416 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:46:47.494874 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-184f499e-e354-4275-8ca2-f4cba48b8416 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:46:47.497676 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-184f499e-e354-4275-8ca2-f4cba48b8416 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:46:47.499992 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-184f499e-e354-4275-8ca2-f4cba48b8416 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] 10.222.0.26 "GET /compute/v2.1/servers/85efeeae-0648-4a33-bc74-c0f0d25880aa" status: 200 len: 1721 microversion: 2.20 time: 0.207432 Oct 19 00:46:47.501056 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 593/708] 10.222.0.26 () {60 vars in 1277 bytes} [Wed Oct 19 00:46:47 2022] GET /compute/v2.1/servers/85efeeae-0648-4a33-bc74-c0f0d25880aa => generated 1721 bytes in 210 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 19 00:46:48.527789 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-caf4c51b-e592-475c-b31d-ea357fa9e550 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:46:48.528529 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-caf4c51b-e592-475c-b31d-ea357fa9e550 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] [instance: 85efeeae-0648-4a33-bc74-c0f0d25880aa] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:46:48.535977 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-caf4c51b-e592-475c-b31d-ea357fa9e550 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:46:48.536631 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-caf4c51b-e592-475c-b31d-ea357fa9e550 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:46:48.537121 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-caf4c51b-e592-475c-b31d-ea357fa9e550 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:46:48.537922 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-caf4c51b-e592-475c-b31d-ea357fa9e550 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:46:48.538419 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-caf4c51b-e592-475c-b31d-ea357fa9e550 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:46:48.539009 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-caf4c51b-e592-475c-b31d-ea357fa9e550 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:46:48.732226 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-caf4c51b-e592-475c-b31d-ea357fa9e550 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:46:48.748804 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-caf4c51b-e592-475c-b31d-ea357fa9e550 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:46:48.751985 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-caf4c51b-e592-475c-b31d-ea357fa9e550 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:46:48.754275 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-caf4c51b-e592-475c-b31d-ea357fa9e550 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] 10.222.0.26 "GET /compute/v2.1/servers/85efeeae-0648-4a33-bc74-c0f0d25880aa" status: 200 len: 1721 microversion: 2.20 time: 0.230441 Oct 19 00:46:48.756280 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 594/709] 10.222.0.26 () {60 vars in 1277 bytes} [Wed Oct 19 00:46:48 2022] GET /compute/v2.1/servers/85efeeae-0648-4a33-bc74-c0f0d25880aa => generated 1721 bytes in 233 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 19 00:46:49.788769 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-f2eb1287-f7ed-4d74-b377-0156b81ca62f tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:46:49.789486 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-f2eb1287-f7ed-4d74-b377-0156b81ca62f tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] [instance: 85efeeae-0648-4a33-bc74-c0f0d25880aa] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:46:49.796521 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-f2eb1287-f7ed-4d74-b377-0156b81ca62f tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:46:49.796911 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-f2eb1287-f7ed-4d74-b377-0156b81ca62f tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:46:49.797386 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-f2eb1287-f7ed-4d74-b377-0156b81ca62f tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:46:49.798112 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-f2eb1287-f7ed-4d74-b377-0156b81ca62f tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:46:49.798603 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-f2eb1287-f7ed-4d74-b377-0156b81ca62f tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:46:49.799035 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-f2eb1287-f7ed-4d74-b377-0156b81ca62f tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:46:49.979240 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-f2eb1287-f7ed-4d74-b377-0156b81ca62f tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:46:49.988329 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-f2eb1287-f7ed-4d74-b377-0156b81ca62f tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:46:49.989827 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-f2eb1287-f7ed-4d74-b377-0156b81ca62f tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:46:49.991080 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-f2eb1287-f7ed-4d74-b377-0156b81ca62f tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] 10.222.0.26 "GET /compute/v2.1/servers/85efeeae-0648-4a33-bc74-c0f0d25880aa" status: 200 len: 1664 microversion: 2.20 time: 0.206944 Oct 19 00:46:49.991651 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 595/710] 10.222.0.26 () {60 vars in 1277 bytes} [Wed Oct 19 00:46:49 2022] GET /compute/v2.1/servers/85efeeae-0648-4a33-bc74-c0f0d25880aa => generated 1664 bytes in 208 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 19 00:46:50.016911 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-949e639c-8cab-4cf0-80cf-fb2e25c6c9a1 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "9355f4e2-36f8-4fa1-ae12-6cecac343f48"}} {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 19 00:46:50.019422 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-949e639c-8cab-4cf0-80cf-fb2e25c6c9a1 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] [instance: 85efeeae-0648-4a33-bc74-c0f0d25880aa] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:46:50.026754 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-949e639c-8cab-4cf0-80cf-fb2e25c6c9a1 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:46:50.027244 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-949e639c-8cab-4cf0-80cf-fb2e25c6c9a1 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:46:50.027677 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-949e639c-8cab-4cf0-80cf-fb2e25c6c9a1 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:46:50.028382 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-949e639c-8cab-4cf0-80cf-fb2e25c6c9a1 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:46:50.028833 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-949e639c-8cab-4cf0-80cf-fb2e25c6c9a1 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:46:50.029259 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-949e639c-8cab-4cf0-80cf-fb2e25c6c9a1 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:46:50.372499 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-949e639c-8cab-4cf0-80cf-fb2e25c6c9a1 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] 10.222.0.26 "POST /compute/v2.1/servers/85efeeae-0648-4a33-bc74-c0f0d25880aa/os-volume_attachments" status: 200 len: 188 microversion: 2.20 time: 0.359324 Oct 19 00:46:50.373199 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 596/711] 10.222.0.26 () {62 vars in 1364 bytes} [Wed Oct 19 00:46:50 2022] POST /compute/v2.1/servers/85efeeae-0648-4a33-bc74-c0f0d25880aa/os-volume_attachments => generated 188 bytes in 361 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Oct 19 00:46:50.539709 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-9ad9aebc-5032-4f64-9f1c-d955ab9823fe tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:46:50.540577 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-9ad9aebc-5032-4f64-9f1c-d955ab9823fe tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] [instance: 85efeeae-0648-4a33-bc74-c0f0d25880aa] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:46:50.552872 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-9ad9aebc-5032-4f64-9f1c-d955ab9823fe tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:46:50.553962 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-9ad9aebc-5032-4f64-9f1c-d955ab9823fe tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:46:50.554950 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-9ad9aebc-5032-4f64-9f1c-d955ab9823fe tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:46:50.556262 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-9ad9aebc-5032-4f64-9f1c-d955ab9823fe tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:46:50.557610 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-9ad9aebc-5032-4f64-9f1c-d955ab9823fe tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:46:50.557867 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-9ad9aebc-5032-4f64-9f1c-d955ab9823fe tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:46:50.876344 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-9ad9aebc-5032-4f64-9f1c-d955ab9823fe tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] 10.222.0.26 "DELETE /compute/v2.1/servers/85efeeae-0648-4a33-bc74-c0f0d25880aa/os-volume_attachments/9355f4e2-36f8-4fa1-ae12-6cecac343f48" status: 202 len: 0 microversion: 2.20 time: 0.345136 Oct 19 00:46:50.882691 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 597/712] 10.222.0.26 () {60 vars in 1457 bytes} [Wed Oct 19 00:46:50 2022] DELETE /compute/v2.1/servers/85efeeae-0648-4a33-bc74-c0f0d25880aa/os-volume_attachments/9355f4e2-36f8-4fa1-ae12-6cecac343f48 => generated 0 bytes in 351 msecs (HTTP/1.1 202) 9 headers in 363 bytes (1 switches on core 0) Oct 19 00:46:50.926143 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [req-17e41bee-6112-44b0-96fe-8662517bf084 req-5bfd5601-d0e6-41a6-950b-d8e5c138301e service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "85efeeae-0648-4a33-bc74-c0f0d25880aa", "name": "network-vif-unplugged", "status": "completed", "tag": "ba933942-134c-497e-8475-ff284eee237e"}]} {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 19 00:46:50.939855 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-17e41bee-6112-44b0-96fe-8662517bf084 req-5bfd5601-d0e6-41a6-950b-d8e5c138301e service nova] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:46:50.940661 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-17e41bee-6112-44b0-96fe-8662517bf084 req-5bfd5601-d0e6-41a6-950b-d8e5c138301e service nova] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:46:50.941101 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-17e41bee-6112-44b0-96fe-8662517bf084 req-5bfd5601-d0e6-41a6-950b-d8e5c138301e service nova] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:46:50.953762 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.compute.server_external_events [req-17e41bee-6112-44b0-96fe-8662517bf084 req-5bfd5601-d0e6-41a6-950b-d8e5c138301e service nova] Unable to find a host for instance 85efeeae-0648-4a33-bc74-c0f0d25880aa. Dropping event network-vif-unplugged {{(pid=125641) create /opt/stack/nova/nova/api/openstack/compute/server_external_events.py:144}} Oct 19 00:46:50.954751 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [req-17e41bee-6112-44b0-96fe-8662517bf084 req-5bfd5601-d0e6-41a6-950b-d8e5c138301e service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 207 len: 182 microversion: 2.1 time: 0.035012 Oct 19 00:46:50.955427 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 598/713] 10.222.0.26 () {64 vars in 1317 bytes} [Wed Oct 19 00:46:50 2022] POST /compute/v2.1/os-server-external-events => generated 182 bytes in 37 msecs (HTTP/1.1 207) 9 headers in 367 bytes (1 switches on core 0) Oct 19 00:46:50.960565 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-dfd9dff5-cb23-4308-aacc-378d23a61469 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Action: 'action', calling method: >, body: {"unshelve": null} {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 19 00:46:50.961168 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-dfd9dff5-cb23-4308-aacc-378d23a61469 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] [instance: 85efeeae-0648-4a33-bc74-c0f0d25880aa] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:46:50.968067 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-dfd9dff5-cb23-4308-aacc-378d23a61469 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:46:50.968466 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-dfd9dff5-cb23-4308-aacc-378d23a61469 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:46:50.968909 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-dfd9dff5-cb23-4308-aacc-378d23a61469 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:46:50.969883 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-dfd9dff5-cb23-4308-aacc-378d23a61469 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:46:50.970340 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-dfd9dff5-cb23-4308-aacc-378d23a61469 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:46:50.970822 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-dfd9dff5-cb23-4308-aacc-378d23a61469 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:46:51.004088 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-dfd9dff5-cb23-4308-aacc-378d23a61469 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] [instance: 85efeeae-0648-4a33-bc74-c0f0d25880aa] Unshelving instance with old availability_zone "None" to new availability_zone "not provided" and host "None". {{(pid=125641) unshelve /opt/stack/nova/nova/compute/api.py:4593}} Oct 19 00:46:51.081357 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-dfd9dff5-cb23-4308-aacc-378d23a61469 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] 10.222.0.26 "POST /compute/v2.1/servers/85efeeae-0648-4a33-bc74-c0f0d25880aa/action" status: 202 len: 0 microversion: 2.20 time: 0.124691 Oct 19 00:46:51.082707 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 599/714] 10.222.0.26 () {62 vars in 1319 bytes} [Wed Oct 19 00:46:50 2022] POST /compute/v2.1/servers/85efeeae-0648-4a33-bc74-c0f0d25880aa/action => generated 0 bytes in 127 msecs (HTTP/1.1 202) 9 headers in 363 bytes (1 switches on core 0) Oct 19 00:46:51.092347 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-9cf9539d-1b84-420b-88bc-b5f716345a45 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:46:51.092989 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-9cf9539d-1b84-420b-88bc-b5f716345a45 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] [instance: 85efeeae-0648-4a33-bc74-c0f0d25880aa] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:46:51.101953 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-9cf9539d-1b84-420b-88bc-b5f716345a45 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:46:51.102615 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-9cf9539d-1b84-420b-88bc-b5f716345a45 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:46:51.103171 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-9cf9539d-1b84-420b-88bc-b5f716345a45 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:46:51.104015 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-9cf9539d-1b84-420b-88bc-b5f716345a45 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:46:51.104621 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-9cf9539d-1b84-420b-88bc-b5f716345a45 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:46:51.105132 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-9cf9539d-1b84-420b-88bc-b5f716345a45 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:46:51.274900 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-9cf9539d-1b84-420b-88bc-b5f716345a45 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:46:51.286295 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-9cf9539d-1b84-420b-88bc-b5f716345a45 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:46:51.288667 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-9cf9539d-1b84-420b-88bc-b5f716345a45 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:46:51.290013 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-9cf9539d-1b84-420b-88bc-b5f716345a45 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] 10.222.0.26 "GET /compute/v2.1/servers/85efeeae-0648-4a33-bc74-c0f0d25880aa" status: 200 len: 1672 microversion: 2.20 time: 0.201993 Oct 19 00:46:51.290691 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 600/715] 10.222.0.26 () {60 vars in 1277 bytes} [Wed Oct 19 00:46:51 2022] GET /compute/v2.1/servers/85efeeae-0648-4a33-bc74-c0f0d25880aa => generated 1672 bytes in 203 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 19 00:46:52.328813 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-f83c5a13-87c6-4c3f-8c57-ba16ea084f2f tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:46:52.329627 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-f83c5a13-87c6-4c3f-8c57-ba16ea084f2f tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] [instance: 85efeeae-0648-4a33-bc74-c0f0d25880aa] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:46:52.336841 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-f83c5a13-87c6-4c3f-8c57-ba16ea084f2f tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:46:52.337337 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-f83c5a13-87c6-4c3f-8c57-ba16ea084f2f tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:46:52.337779 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-f83c5a13-87c6-4c3f-8c57-ba16ea084f2f tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:46:52.338525 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-f83c5a13-87c6-4c3f-8c57-ba16ea084f2f tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:46:52.339062 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-f83c5a13-87c6-4c3f-8c57-ba16ea084f2f tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:46:52.339496 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-f83c5a13-87c6-4c3f-8c57-ba16ea084f2f tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:46:52.526970 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-f83c5a13-87c6-4c3f-8c57-ba16ea084f2f tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:46:52.544480 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-f83c5a13-87c6-4c3f-8c57-ba16ea084f2f tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:46:52.547994 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-f83c5a13-87c6-4c3f-8c57-ba16ea084f2f tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:46:52.550956 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-f83c5a13-87c6-4c3f-8c57-ba16ea084f2f tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] 10.222.0.26 "GET /compute/v2.1/servers/85efeeae-0648-4a33-bc74-c0f0d25880aa" status: 200 len: 1674 microversion: 2.20 time: 0.225464 Oct 19 00:46:52.552647 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 601/716] 10.222.0.26 () {60 vars in 1277 bytes} [Wed Oct 19 00:46:52 2022] GET /compute/v2.1/servers/85efeeae-0648-4a33-bc74-c0f0d25880aa => generated 1674 bytes in 228 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 19 00:46:53.585772 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-20bcf74c-ef32-418e-a8b2-5f3befb2e265 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:46:53.586924 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-20bcf74c-ef32-418e-a8b2-5f3befb2e265 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] [instance: 85efeeae-0648-4a33-bc74-c0f0d25880aa] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:46:53.594740 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-20bcf74c-ef32-418e-a8b2-5f3befb2e265 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:46:53.595254 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-20bcf74c-ef32-418e-a8b2-5f3befb2e265 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:46:53.595663 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-20bcf74c-ef32-418e-a8b2-5f3befb2e265 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:46:53.596381 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-20bcf74c-ef32-418e-a8b2-5f3befb2e265 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:46:53.596833 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-20bcf74c-ef32-418e-a8b2-5f3befb2e265 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:46:53.597406 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-20bcf74c-ef32-418e-a8b2-5f3befb2e265 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:46:53.774043 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-20bcf74c-ef32-418e-a8b2-5f3befb2e265 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:46:53.784955 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-20bcf74c-ef32-418e-a8b2-5f3befb2e265 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:46:53.787221 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-20bcf74c-ef32-418e-a8b2-5f3befb2e265 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:46:53.789342 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-20bcf74c-ef32-418e-a8b2-5f3befb2e265 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] 10.222.0.26 "GET /compute/v2.1/servers/85efeeae-0648-4a33-bc74-c0f0d25880aa" status: 200 len: 1730 microversion: 2.20 time: 0.209183 Oct 19 00:46:53.791053 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 602/717] 10.222.0.26 () {60 vars in 1277 bytes} [Wed Oct 19 00:46:53 2022] GET /compute/v2.1/servers/85efeeae-0648-4a33-bc74-c0f0d25880aa => generated 1730 bytes in 211 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 19 00:46:54.824139 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-59f1987e-6218-42fd-ac72-e80bf4a1630b tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:46:54.825651 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-59f1987e-6218-42fd-ac72-e80bf4a1630b tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] [instance: 85efeeae-0648-4a33-bc74-c0f0d25880aa] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:46:54.833745 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-59f1987e-6218-42fd-ac72-e80bf4a1630b tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:46:54.834674 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-59f1987e-6218-42fd-ac72-e80bf4a1630b tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:46:54.835567 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-59f1987e-6218-42fd-ac72-e80bf4a1630b tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:46:54.836644 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-59f1987e-6218-42fd-ac72-e80bf4a1630b tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:46:54.837402 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-59f1987e-6218-42fd-ac72-e80bf4a1630b tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:46:54.838097 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-59f1987e-6218-42fd-ac72-e80bf4a1630b tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:46:54.843386 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG dbcounter [-] [125641] Writing DB stats nova_cell1:SELECT=85,nova_cell1:INSERT=6,nova_cell1:UPDATE=9 {{(pid=125641) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 00:46:55.061433 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-59f1987e-6218-42fd-ac72-e80bf4a1630b tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:46:55.072949 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-59f1987e-6218-42fd-ac72-e80bf4a1630b tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:46:55.074869 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-59f1987e-6218-42fd-ac72-e80bf4a1630b tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:46:55.076284 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-59f1987e-6218-42fd-ac72-e80bf4a1630b tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] 10.222.0.26 "GET /compute/v2.1/servers/85efeeae-0648-4a33-bc74-c0f0d25880aa" status: 200 len: 1730 microversion: 2.20 time: 0.256135 Oct 19 00:46:55.077246 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 603/718] 10.222.0.26 () {60 vars in 1277 bytes} [Wed Oct 19 00:46:54 2022] GET /compute/v2.1/servers/85efeeae-0648-4a33-bc74-c0f0d25880aa => generated 1730 bytes in 258 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 19 00:46:56.111762 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-509f8b13-efd0-4f52-b9bc-7a5078987d1e tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:46:56.114906 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-509f8b13-efd0-4f52-b9bc-7a5078987d1e tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] [instance: 85efeeae-0648-4a33-bc74-c0f0d25880aa] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:46:56.129578 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-509f8b13-efd0-4f52-b9bc-7a5078987d1e tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:46:56.130716 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-509f8b13-efd0-4f52-b9bc-7a5078987d1e tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:46:56.131170 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-509f8b13-efd0-4f52-b9bc-7a5078987d1e tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:46:56.131905 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-509f8b13-efd0-4f52-b9bc-7a5078987d1e tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:46:56.132362 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-509f8b13-efd0-4f52-b9bc-7a5078987d1e tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:46:56.132800 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-509f8b13-efd0-4f52-b9bc-7a5078987d1e tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:46:56.332527 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-509f8b13-efd0-4f52-b9bc-7a5078987d1e tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:46:56.343000 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-509f8b13-efd0-4f52-b9bc-7a5078987d1e tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:46:56.345773 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-509f8b13-efd0-4f52-b9bc-7a5078987d1e tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:46:56.347603 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-509f8b13-efd0-4f52-b9bc-7a5078987d1e tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] 10.222.0.26 "GET /compute/v2.1/servers/85efeeae-0648-4a33-bc74-c0f0d25880aa" status: 200 len: 1730 microversion: 2.20 time: 0.244209 Oct 19 00:46:56.349618 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 604/719] 10.222.0.26 () {60 vars in 1277 bytes} [Wed Oct 19 00:46:56 2022] GET /compute/v2.1/servers/85efeeae-0648-4a33-bc74-c0f0d25880aa => generated 1730 bytes in 246 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 19 00:46:57.390204 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-dcabb29d-b804-4351-ae51-d71fc7c4e767 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:46:57.390204 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-dcabb29d-b804-4351-ae51-d71fc7c4e767 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] [instance: 85efeeae-0648-4a33-bc74-c0f0d25880aa] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:46:57.393605 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG dbcounter [-] [125641] Writing DB stats nova_api:SELECT=99,nova_api:UPDATE=1 {{(pid=125641) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 00:46:57.397541 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-dcabb29d-b804-4351-ae51-d71fc7c4e767 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:46:57.398036 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-dcabb29d-b804-4351-ae51-d71fc7c4e767 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:46:57.398513 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-dcabb29d-b804-4351-ae51-d71fc7c4e767 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:46:57.399259 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-dcabb29d-b804-4351-ae51-d71fc7c4e767 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:46:57.399733 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-dcabb29d-b804-4351-ae51-d71fc7c4e767 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:46:57.400162 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-dcabb29d-b804-4351-ae51-d71fc7c4e767 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:46:57.600944 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-dcabb29d-b804-4351-ae51-d71fc7c4e767 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:46:57.619121 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-dcabb29d-b804-4351-ae51-d71fc7c4e767 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:46:57.622401 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-dcabb29d-b804-4351-ae51-d71fc7c4e767 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:46:57.624919 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-dcabb29d-b804-4351-ae51-d71fc7c4e767 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] 10.222.0.26 "GET /compute/v2.1/servers/85efeeae-0648-4a33-bc74-c0f0d25880aa" status: 200 len: 1730 microversion: 2.20 time: 0.239028 Oct 19 00:46:57.638535 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 605/720] 10.222.0.26 () {60 vars in 1277 bytes} [Wed Oct 19 00:46:57 2022] GET /compute/v2.1/servers/85efeeae-0648-4a33-bc74-c0f0d25880aa => generated 1730 bytes in 241 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 19 00:46:58.658800 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-14012cfb-a083-405b-bb88-06bb0202b293 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:46:58.660966 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-14012cfb-a083-405b-bb88-06bb0202b293 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] [instance: 85efeeae-0648-4a33-bc74-c0f0d25880aa] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:46:58.673419 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-14012cfb-a083-405b-bb88-06bb0202b293 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:46:58.674274 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-14012cfb-a083-405b-bb88-06bb0202b293 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:46:58.675090 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-14012cfb-a083-405b-bb88-06bb0202b293 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:46:58.676495 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-14012cfb-a083-405b-bb88-06bb0202b293 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:46:58.678468 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-14012cfb-a083-405b-bb88-06bb0202b293 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:46:58.679068 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-14012cfb-a083-405b-bb88-06bb0202b293 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:46:58.925354 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-14012cfb-a083-405b-bb88-06bb0202b293 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:46:58.939821 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-14012cfb-a083-405b-bb88-06bb0202b293 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:46:58.943624 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-14012cfb-a083-405b-bb88-06bb0202b293 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:46:58.946200 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-14012cfb-a083-405b-bb88-06bb0202b293 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] 10.222.0.26 "GET /compute/v2.1/servers/85efeeae-0648-4a33-bc74-c0f0d25880aa" status: 200 len: 1730 microversion: 2.20 time: 0.296018 Oct 19 00:46:58.950535 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 606/721] 10.222.0.26 () {60 vars in 1277 bytes} [Wed Oct 19 00:46:58 2022] GET /compute/v2.1/servers/85efeeae-0648-4a33-bc74-c0f0d25880aa => generated 1730 bytes in 299 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 19 00:46:59.988206 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-1a345001-46e4-4914-9b0d-899b81e2aa2c tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:46:59.990738 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-1a345001-46e4-4914-9b0d-899b81e2aa2c tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] [instance: 85efeeae-0648-4a33-bc74-c0f0d25880aa] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:47:00.002909 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-1a345001-46e4-4914-9b0d-899b81e2aa2c tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:47:00.003829 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-1a345001-46e4-4914-9b0d-899b81e2aa2c tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:47:00.004670 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-1a345001-46e4-4914-9b0d-899b81e2aa2c tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:47:00.006330 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-1a345001-46e4-4914-9b0d-899b81e2aa2c tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:47:00.007154 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-1a345001-46e4-4914-9b0d-899b81e2aa2c tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:47:00.007944 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-1a345001-46e4-4914-9b0d-899b81e2aa2c tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:47:00.299083 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-1a345001-46e4-4914-9b0d-899b81e2aa2c tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:47:00.309735 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-1a345001-46e4-4914-9b0d-899b81e2aa2c tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:47:00.338757 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-1a345001-46e4-4914-9b0d-899b81e2aa2c tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:47:00.338757 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-1a345001-46e4-4914-9b0d-899b81e2aa2c tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] 10.222.0.26 "GET /compute/v2.1/servers/85efeeae-0648-4a33-bc74-c0f0d25880aa" status: 200 len: 1730 microversion: 2.20 time: 0.332523 Oct 19 00:47:00.338757 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 607/722] 10.222.0.26 () {60 vars in 1277 bytes} [Wed Oct 19 00:46:59 2022] GET /compute/v2.1/servers/85efeeae-0648-4a33-bc74-c0f0d25880aa => generated 1730 bytes in 334 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 19 00:47:01.359858 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-af20dd7a-f1c9-457e-a084-82cf8c86ab00 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:47:01.361059 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-af20dd7a-f1c9-457e-a084-82cf8c86ab00 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] [instance: 85efeeae-0648-4a33-bc74-c0f0d25880aa] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:47:01.369698 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-af20dd7a-f1c9-457e-a084-82cf8c86ab00 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:47:01.370467 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-af20dd7a-f1c9-457e-a084-82cf8c86ab00 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:47:01.371156 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-af20dd7a-f1c9-457e-a084-82cf8c86ab00 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:47:01.372161 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-af20dd7a-f1c9-457e-a084-82cf8c86ab00 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:47:01.372934 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-af20dd7a-f1c9-457e-a084-82cf8c86ab00 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:47:01.373504 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-af20dd7a-f1c9-457e-a084-82cf8c86ab00 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:47:01.613841 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-af20dd7a-f1c9-457e-a084-82cf8c86ab00 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:47:01.624006 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-af20dd7a-f1c9-457e-a084-82cf8c86ab00 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:47:01.625576 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-af20dd7a-f1c9-457e-a084-82cf8c86ab00 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:47:01.626862 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-af20dd7a-f1c9-457e-a084-82cf8c86ab00 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] 10.222.0.26 "GET /compute/v2.1/servers/85efeeae-0648-4a33-bc74-c0f0d25880aa" status: 200 len: 1730 microversion: 2.20 time: 0.271985 Oct 19 00:47:01.627437 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 608/723] 10.222.0.26 () {60 vars in 1277 bytes} [Wed Oct 19 00:47:01 2022] GET /compute/v2.1/servers/85efeeae-0648-4a33-bc74-c0f0d25880aa => generated 1730 bytes in 274 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 19 00:47:02.680630 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-f728cebc-43b0-4522-a90f-8fb1221dd19b tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:47:02.681824 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-f728cebc-43b0-4522-a90f-8fb1221dd19b tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] [instance: 85efeeae-0648-4a33-bc74-c0f0d25880aa] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:47:02.689303 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-f728cebc-43b0-4522-a90f-8fb1221dd19b tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:47:02.689869 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-f728cebc-43b0-4522-a90f-8fb1221dd19b tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:47:02.690315 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-f728cebc-43b0-4522-a90f-8fb1221dd19b tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:47:02.691143 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-f728cebc-43b0-4522-a90f-8fb1221dd19b tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:47:02.691637 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-f728cebc-43b0-4522-a90f-8fb1221dd19b tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:47:02.692128 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-f728cebc-43b0-4522-a90f-8fb1221dd19b tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:47:02.875541 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-f728cebc-43b0-4522-a90f-8fb1221dd19b tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:47:02.886220 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-f728cebc-43b0-4522-a90f-8fb1221dd19b tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:47:02.887923 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-f728cebc-43b0-4522-a90f-8fb1221dd19b tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:47:02.889152 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-f728cebc-43b0-4522-a90f-8fb1221dd19b tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] 10.222.0.26 "GET /compute/v2.1/servers/85efeeae-0648-4a33-bc74-c0f0d25880aa" status: 200 len: 1730 microversion: 2.20 time: 0.213922 Oct 19 00:47:02.889733 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 609/724] 10.222.0.26 () {60 vars in 1277 bytes} [Wed Oct 19 00:47:02 2022] GET /compute/v2.1/servers/85efeeae-0648-4a33-bc74-c0f0d25880aa => generated 1730 bytes in 215 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 19 00:47:03.915855 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-6a13ca99-aea5-44f3-a6ae-e826227343df tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:47:03.916707 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-6a13ca99-aea5-44f3-a6ae-e826227343df tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] [instance: 85efeeae-0648-4a33-bc74-c0f0d25880aa] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:47:03.923714 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-6a13ca99-aea5-44f3-a6ae-e826227343df tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:47:03.924200 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-6a13ca99-aea5-44f3-a6ae-e826227343df tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:47:03.924638 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-6a13ca99-aea5-44f3-a6ae-e826227343df tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:47:03.925347 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-6a13ca99-aea5-44f3-a6ae-e826227343df tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:47:03.925801 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-6a13ca99-aea5-44f3-a6ae-e826227343df tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:47:03.926224 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-6a13ca99-aea5-44f3-a6ae-e826227343df tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:47:04.078856 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-6a13ca99-aea5-44f3-a6ae-e826227343df tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:47:04.088131 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-6a13ca99-aea5-44f3-a6ae-e826227343df tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:47:04.089602 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-6a13ca99-aea5-44f3-a6ae-e826227343df tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:47:04.090859 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-6a13ca99-aea5-44f3-a6ae-e826227343df tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] 10.222.0.26 "GET /compute/v2.1/servers/85efeeae-0648-4a33-bc74-c0f0d25880aa" status: 200 len: 1730 microversion: 2.20 time: 0.179677 Oct 19 00:47:04.091440 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 610/725] 10.222.0.26 () {60 vars in 1277 bytes} [Wed Oct 19 00:47:03 2022] GET /compute/v2.1/servers/85efeeae-0648-4a33-bc74-c0f0d25880aa => generated 1730 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 19 00:47:05.117919 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-a2089929-4b12-4ffc-b947-de676e120791 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:47:05.118873 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-a2089929-4b12-4ffc-b947-de676e120791 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] [instance: 85efeeae-0648-4a33-bc74-c0f0d25880aa] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:47:05.125677 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-a2089929-4b12-4ffc-b947-de676e120791 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:47:05.126170 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-a2089929-4b12-4ffc-b947-de676e120791 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:47:05.126656 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-a2089929-4b12-4ffc-b947-de676e120791 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:47:05.127369 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-a2089929-4b12-4ffc-b947-de676e120791 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:47:05.127825 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-a2089929-4b12-4ffc-b947-de676e120791 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:47:05.128253 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-a2089929-4b12-4ffc-b947-de676e120791 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:47:05.299003 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-a2089929-4b12-4ffc-b947-de676e120791 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:47:05.308121 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-a2089929-4b12-4ffc-b947-de676e120791 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:47:05.309600 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-a2089929-4b12-4ffc-b947-de676e120791 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:47:05.310843 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-a2089929-4b12-4ffc-b947-de676e120791 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] 10.222.0.26 "GET /compute/v2.1/servers/85efeeae-0648-4a33-bc74-c0f0d25880aa" status: 200 len: 1730 microversion: 2.20 time: 0.197279 Oct 19 00:47:05.311425 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 611/726] 10.222.0.26 () {60 vars in 1277 bytes} [Wed Oct 19 00:47:05 2022] GET /compute/v2.1/servers/85efeeae-0648-4a33-bc74-c0f0d25880aa => generated 1730 bytes in 198 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 19 00:47:06.338176 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-0a375e9c-e76b-4b83-9c16-fd3cdbcba76b tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:47:06.339326 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-0a375e9c-e76b-4b83-9c16-fd3cdbcba76b tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] [instance: 85efeeae-0648-4a33-bc74-c0f0d25880aa] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:47:06.346383 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-0a375e9c-e76b-4b83-9c16-fd3cdbcba76b tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:47:06.347044 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-0a375e9c-e76b-4b83-9c16-fd3cdbcba76b tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:47:06.347483 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-0a375e9c-e76b-4b83-9c16-fd3cdbcba76b tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:47:06.348277 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-0a375e9c-e76b-4b83-9c16-fd3cdbcba76b tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:47:06.348797 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-0a375e9c-e76b-4b83-9c16-fd3cdbcba76b tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:47:06.349282 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-0a375e9c-e76b-4b83-9c16-fd3cdbcba76b tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:47:06.537357 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-0a375e9c-e76b-4b83-9c16-fd3cdbcba76b tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:47:06.549655 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-0a375e9c-e76b-4b83-9c16-fd3cdbcba76b tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:47:06.553089 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-0a375e9c-e76b-4b83-9c16-fd3cdbcba76b tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:47:06.555043 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-0a375e9c-e76b-4b83-9c16-fd3cdbcba76b tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] 10.222.0.26 "GET /compute/v2.1/servers/85efeeae-0648-4a33-bc74-c0f0d25880aa" status: 200 len: 1730 microversion: 2.20 time: 0.221466 Oct 19 00:47:06.555785 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 612/727] 10.222.0.26 () {60 vars in 1277 bytes} [Wed Oct 19 00:47:06 2022] GET /compute/v2.1/servers/85efeeae-0648-4a33-bc74-c0f0d25880aa => generated 1730 bytes in 223 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 19 00:47:07.588237 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-0d9d53cf-2206-421f-ae1e-e68beb497f60 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:47:07.589341 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-0d9d53cf-2206-421f-ae1e-e68beb497f60 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] [instance: 85efeeae-0648-4a33-bc74-c0f0d25880aa] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:47:07.597836 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-0d9d53cf-2206-421f-ae1e-e68beb497f60 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:47:07.598581 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-0d9d53cf-2206-421f-ae1e-e68beb497f60 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:47:07.599144 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-0d9d53cf-2206-421f-ae1e-e68beb497f60 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:47:07.600029 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-0d9d53cf-2206-421f-ae1e-e68beb497f60 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:47:07.600689 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-0d9d53cf-2206-421f-ae1e-e68beb497f60 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:47:07.601702 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-0d9d53cf-2206-421f-ae1e-e68beb497f60 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:47:07.791403 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-0d9d53cf-2206-421f-ae1e-e68beb497f60 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:47:07.801107 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-0d9d53cf-2206-421f-ae1e-e68beb497f60 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:47:07.803340 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-0d9d53cf-2206-421f-ae1e-e68beb497f60 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:47:07.804690 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-0d9d53cf-2206-421f-ae1e-e68beb497f60 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] 10.222.0.26 "GET /compute/v2.1/servers/85efeeae-0648-4a33-bc74-c0f0d25880aa" status: 200 len: 1730 microversion: 2.20 time: 0.221096 Oct 19 00:47:07.805350 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 613/728] 10.222.0.26 () {60 vars in 1277 bytes} [Wed Oct 19 00:47:07 2022] GET /compute/v2.1/servers/85efeeae-0648-4a33-bc74-c0f0d25880aa => generated 1730 bytes in 223 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 19 00:47:08.830915 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-be1e3939-368f-4f0d-846a-9bf4e4b3189a tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:47:08.831767 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-be1e3939-368f-4f0d-846a-9bf4e4b3189a tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] [instance: 85efeeae-0648-4a33-bc74-c0f0d25880aa] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:47:08.838922 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-be1e3939-368f-4f0d-846a-9bf4e4b3189a tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:47:08.839408 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-be1e3939-368f-4f0d-846a-9bf4e4b3189a tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:47:08.839847 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-be1e3939-368f-4f0d-846a-9bf4e4b3189a tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:47:08.840558 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-be1e3939-368f-4f0d-846a-9bf4e4b3189a tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:47:08.841011 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-be1e3939-368f-4f0d-846a-9bf4e4b3189a tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:47:08.841437 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-be1e3939-368f-4f0d-846a-9bf4e4b3189a tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:47:09.026545 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-be1e3939-368f-4f0d-846a-9bf4e4b3189a tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:47:09.035398 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-be1e3939-368f-4f0d-846a-9bf4e4b3189a tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:47:09.037470 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-be1e3939-368f-4f0d-846a-9bf4e4b3189a tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:47:09.038793 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-be1e3939-368f-4f0d-846a-9bf4e4b3189a tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] 10.222.0.26 "GET /compute/v2.1/servers/85efeeae-0648-4a33-bc74-c0f0d25880aa" status: 200 len: 1730 microversion: 2.20 time: 0.212141 Oct 19 00:47:09.039444 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 614/729] 10.222.0.26 () {60 vars in 1277 bytes} [Wed Oct 19 00:47:08 2022] GET /compute/v2.1/servers/85efeeae-0648-4a33-bc74-c0f0d25880aa => generated 1730 bytes in 213 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 19 00:47:09.122842 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [req-13c1006f-6e5a-4e5b-9d5a-21b79421adab req-8a5cc1dc-49b6-4372-bc20-bfdcbff0c905 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "85efeeae-0648-4a33-bc74-c0f0d25880aa", "name": "network-vif-plugged", "status": "completed", "tag": "ba933942-134c-497e-8475-ff284eee237e"}]} {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 19 00:47:09.137943 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-13c1006f-6e5a-4e5b-9d5a-21b79421adab req-8a5cc1dc-49b6-4372-bc20-bfdcbff0c905 service nova] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:47:09.138829 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-13c1006f-6e5a-4e5b-9d5a-21b79421adab req-8a5cc1dc-49b6-4372-bc20-bfdcbff0c905 service nova] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:47:09.139615 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-13c1006f-6e5a-4e5b-9d5a-21b79421adab req-8a5cc1dc-49b6-4372-bc20-bfdcbff0c905 service nova] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:47:09.162669 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.compute.server_external_events [req-13c1006f-6e5a-4e5b-9d5a-21b79421adab req-8a5cc1dc-49b6-4372-bc20-bfdcbff0c905 service nova] Creating event network-vif-plugged:ba933942-134c-497e-8475-ff284eee237e for instance 85efeeae-0648-4a33-bc74-c0f0d25880aa on hv-cinder-maste Oct 19 00:47:09.172267 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [req-13c1006f-6e5a-4e5b-9d5a-21b79421adab req-8a5cc1dc-49b6-4372-bc20-bfdcbff0c905 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.054027 Oct 19 00:47:09.173380 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 615/730] 10.222.0.26 () {64 vars in 1317 bytes} [Wed Oct 19 00:47:09 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 19 00:47:10.067101 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-cc0b5dde-e4d1-4a52-b0bc-1cba6c5e0cdb tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:47:10.067955 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-cc0b5dde-e4d1-4a52-b0bc-1cba6c5e0cdb tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] [instance: 85efeeae-0648-4a33-bc74-c0f0d25880aa] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:47:10.075052 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-cc0b5dde-e4d1-4a52-b0bc-1cba6c5e0cdb tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:47:10.075549 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-cc0b5dde-e4d1-4a52-b0bc-1cba6c5e0cdb tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:47:10.075998 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-cc0b5dde-e4d1-4a52-b0bc-1cba6c5e0cdb tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:47:10.076730 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-cc0b5dde-e4d1-4a52-b0bc-1cba6c5e0cdb tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:47:10.077198 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-cc0b5dde-e4d1-4a52-b0bc-1cba6c5e0cdb tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:47:10.077628 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-cc0b5dde-e4d1-4a52-b0bc-1cba6c5e0cdb tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:47:10.251976 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-cc0b5dde-e4d1-4a52-b0bc-1cba6c5e0cdb tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:47:10.267545 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-cc0b5dde-e4d1-4a52-b0bc-1cba6c5e0cdb tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:47:10.271503 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-cc0b5dde-e4d1-4a52-b0bc-1cba6c5e0cdb tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:47:10.273801 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-cc0b5dde-e4d1-4a52-b0bc-1cba6c5e0cdb tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] 10.222.0.26 "GET /compute/v2.1/servers/85efeeae-0648-4a33-bc74-c0f0d25880aa" status: 200 len: 1730 microversion: 2.20 time: 0.211523 Oct 19 00:47:10.274669 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 616/731] 10.222.0.26 () {60 vars in 1277 bytes} [Wed Oct 19 00:47:10 2022] GET /compute/v2.1/servers/85efeeae-0648-4a33-bc74-c0f0d25880aa => generated 1730 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 19 00:47:10.373731 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-cf998b8f-31ce-4dbd-8197-6604e44961ef tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:47:10.374367 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-cf998b8f-31ce-4dbd-8197-6604e44961ef tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] [instance: 0b7c19ee-6d89-4855-bf4b-d35b65bb170b] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:47:10.382391 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-cf998b8f-31ce-4dbd-8197-6604e44961ef tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:47:10.383025 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-cf998b8f-31ce-4dbd-8197-6604e44961ef tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:47:10.383583 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-cf998b8f-31ce-4dbd-8197-6604e44961ef tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:47:10.384493 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-cf998b8f-31ce-4dbd-8197-6604e44961ef tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:47:10.385365 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-cf998b8f-31ce-4dbd-8197-6604e44961ef tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:47:10.386169 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-cf998b8f-31ce-4dbd-8197-6604e44961ef tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:47:10.417694 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-cf998b8f-31ce-4dbd-8197-6604e44961ef tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] [instance: 0b7c19ee-6d89-4855-bf4b-d35b65bb170b] Going to try to terminate instance {{(pid=125641) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 19 00:47:10.518317 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-cf998b8f-31ce-4dbd-8197-6604e44961ef tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] 10.222.0.26 "DELETE /compute/v2.1/servers/0b7c19ee-6d89-4855-bf4b-d35b65bb170b" status: 204 len: 0 microversion: 2.1 time: 0.148513 Oct 19 00:47:10.519066 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 617/732] 10.222.0.26 () {58 vars in 1239 bytes} [Wed Oct 19 00:47:10 2022] DELETE /compute/v2.1/servers/0b7c19ee-6d89-4855-bf4b-d35b65bb170b => generated 0 bytes in 150 msecs (HTTP/1.1 204) 9 headers in 363 bytes (2 switches on core 0) Oct 19 00:47:10.528400 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-8f22ef4a-1311-44bf-8074-192a5fc2cabe tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:47:10.529027 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-8f22ef4a-1311-44bf-8074-192a5fc2cabe tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] [instance: 0b7c19ee-6d89-4855-bf4b-d35b65bb170b] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:47:10.536509 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-8f22ef4a-1311-44bf-8074-192a5fc2cabe tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:47:10.536509 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-8f22ef4a-1311-44bf-8074-192a5fc2cabe tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:47:10.537012 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-8f22ef4a-1311-44bf-8074-192a5fc2cabe tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:47:10.537732 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-8f22ef4a-1311-44bf-8074-192a5fc2cabe tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:47:10.538186 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-8f22ef4a-1311-44bf-8074-192a5fc2cabe tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:47:10.538649 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-8f22ef4a-1311-44bf-8074-192a5fc2cabe tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:47:10.694888 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-8f22ef4a-1311-44bf-8074-192a5fc2cabe tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f738e67d8c2340028da1254b60d7810e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8684504b0eff415b9063cebae21ca667', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:47:10.697409 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG dbcounter [-] [125641] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=125641) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 00:47:10.705421 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-8f22ef4a-1311-44bf-8074-192a5fc2cabe tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] 10.222.0.26 "GET /compute/v2.1/servers/0b7c19ee-6d89-4855-bf4b-d35b65bb170b" status: 200 len: 1577 microversion: 2.1 time: 0.181520 Oct 19 00:47:10.706006 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 618/733] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:47:10 2022] GET /compute/v2.1/servers/0b7c19ee-6d89-4855-bf4b-d35b65bb170b => generated 1577 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 19 00:47:11.316684 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-f4962144-550a-4eea-bfcc-91034f36c433 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:47:11.317490 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-f4962144-550a-4eea-bfcc-91034f36c433 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] [instance: 85efeeae-0648-4a33-bc74-c0f0d25880aa] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:47:11.326379 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-f4962144-550a-4eea-bfcc-91034f36c433 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:47:11.326903 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-f4962144-550a-4eea-bfcc-91034f36c433 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:47:11.327343 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-f4962144-550a-4eea-bfcc-91034f36c433 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:47:11.328071 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-f4962144-550a-4eea-bfcc-91034f36c433 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:47:11.328534 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-f4962144-550a-4eea-bfcc-91034f36c433 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:47:11.328965 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-f4962144-550a-4eea-bfcc-91034f36c433 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:47:11.510310 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-f4962144-550a-4eea-bfcc-91034f36c433 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:47:11.519496 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-f4962144-550a-4eea-bfcc-91034f36c433 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:47:11.521688 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-f4962144-550a-4eea-bfcc-91034f36c433 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:47:11.522999 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-f4962144-550a-4eea-bfcc-91034f36c433 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] 10.222.0.26 "GET /compute/v2.1/servers/85efeeae-0648-4a33-bc74-c0f0d25880aa" status: 200 len: 1730 microversion: 2.20 time: 0.210488 Oct 19 00:47:11.523579 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 619/734] 10.222.0.26 () {60 vars in 1277 bytes} [Wed Oct 19 00:47:11 2022] GET /compute/v2.1/servers/85efeeae-0648-4a33-bc74-c0f0d25880aa => generated 1730 bytes in 212 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 19 00:47:11.732238 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-ce14650e-c018-4ae1-9fc2-abf21f53789c tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:47:11.732924 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-ce14650e-c018-4ae1-9fc2-abf21f53789c tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] [instance: 0b7c19ee-6d89-4855-bf4b-d35b65bb170b] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:47:11.739810 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ce14650e-c018-4ae1-9fc2-abf21f53789c tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:47:11.740389 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ce14650e-c018-4ae1-9fc2-abf21f53789c tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:47:11.740953 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ce14650e-c018-4ae1-9fc2-abf21f53789c tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:47:11.741745 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ce14650e-c018-4ae1-9fc2-abf21f53789c tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:47:11.742317 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ce14650e-c018-4ae1-9fc2-abf21f53789c tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:47:11.742822 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ce14650e-c018-4ae1-9fc2-abf21f53789c tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:47:11.901069 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-ce14650e-c018-4ae1-9fc2-abf21f53789c tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f738e67d8c2340028da1254b60d7810e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8684504b0eff415b9063cebae21ca667', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:47:11.910108 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-ce14650e-c018-4ae1-9fc2-abf21f53789c tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] 10.222.0.26 "GET /compute/v2.1/servers/0b7c19ee-6d89-4855-bf4b-d35b65bb170b" status: 200 len: 1577 microversion: 2.1 time: 0.181763 Oct 19 00:47:11.910745 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 620/735] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:47:11 2022] GET /compute/v2.1/servers/0b7c19ee-6d89-4855-bf4b-d35b65bb170b => generated 1577 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 19 00:47:12.555801 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-4fc675e9-3251-42a9-89fd-ac37e1a554a8 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:47:12.556728 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-4fc675e9-3251-42a9-89fd-ac37e1a554a8 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] [instance: 85efeeae-0648-4a33-bc74-c0f0d25880aa] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:47:12.566412 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-4fc675e9-3251-42a9-89fd-ac37e1a554a8 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:47:12.567682 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-4fc675e9-3251-42a9-89fd-ac37e1a554a8 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:47:12.568194 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-4fc675e9-3251-42a9-89fd-ac37e1a554a8 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:47:12.569097 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-4fc675e9-3251-42a9-89fd-ac37e1a554a8 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:47:12.570150 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-4fc675e9-3251-42a9-89fd-ac37e1a554a8 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:47:12.571139 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-4fc675e9-3251-42a9-89fd-ac37e1a554a8 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:47:12.727624 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-4fc675e9-3251-42a9-89fd-ac37e1a554a8 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:47:12.743322 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-4fc675e9-3251-42a9-89fd-ac37e1a554a8 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:47:12.746518 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-4fc675e9-3251-42a9-89fd-ac37e1a554a8 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:47:12.748840 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-4fc675e9-3251-42a9-89fd-ac37e1a554a8 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] 10.222.0.26 "GET /compute/v2.1/servers/85efeeae-0648-4a33-bc74-c0f0d25880aa" status: 200 len: 1730 microversion: 2.20 time: 0.200857 Oct 19 00:47:12.749938 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 621/736] 10.222.0.26 () {60 vars in 1277 bytes} [Wed Oct 19 00:47:12 2022] GET /compute/v2.1/servers/85efeeae-0648-4a33-bc74-c0f0d25880aa => generated 1730 bytes in 203 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 19 00:47:12.953015 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-318fb583-63b4-449a-9768-847cc0a0a6d7 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:47:12.953981 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-318fb583-63b4-449a-9768-847cc0a0a6d7 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] [instance: 0b7c19ee-6d89-4855-bf4b-d35b65bb170b] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:47:12.961441 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-318fb583-63b4-449a-9768-847cc0a0a6d7 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:47:12.961944 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-318fb583-63b4-449a-9768-847cc0a0a6d7 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:47:12.962369 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-318fb583-63b4-449a-9768-847cc0a0a6d7 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:47:12.963121 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-318fb583-63b4-449a-9768-847cc0a0a6d7 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:47:12.963575 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-318fb583-63b4-449a-9768-847cc0a0a6d7 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:47:12.964006 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-318fb583-63b4-449a-9768-847cc0a0a6d7 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:47:13.139671 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-318fb583-63b4-449a-9768-847cc0a0a6d7 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f738e67d8c2340028da1254b60d7810e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8684504b0eff415b9063cebae21ca667', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:47:13.149692 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-318fb583-63b4-449a-9768-847cc0a0a6d7 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] 10.222.0.26 "GET /compute/v2.1/servers/0b7c19ee-6d89-4855-bf4b-d35b65bb170b" status: 200 len: 1577 microversion: 2.1 time: 0.203565 Oct 19 00:47:13.150416 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 622/737] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:47:12 2022] GET /compute/v2.1/servers/0b7c19ee-6d89-4855-bf4b-d35b65bb170b => generated 1577 bytes in 205 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 19 00:47:13.788444 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-222cb88a-cef1-43f8-bcab-07ef9b7ea556 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:47:13.789168 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-222cb88a-cef1-43f8-bcab-07ef9b7ea556 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] [instance: 85efeeae-0648-4a33-bc74-c0f0d25880aa] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:47:13.796001 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-222cb88a-cef1-43f8-bcab-07ef9b7ea556 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:47:13.796498 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-222cb88a-cef1-43f8-bcab-07ef9b7ea556 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:47:13.796931 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-222cb88a-cef1-43f8-bcab-07ef9b7ea556 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:47:13.797656 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-222cb88a-cef1-43f8-bcab-07ef9b7ea556 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:47:13.798115 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-222cb88a-cef1-43f8-bcab-07ef9b7ea556 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:47:13.798582 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-222cb88a-cef1-43f8-bcab-07ef9b7ea556 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:47:13.987813 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-222cb88a-cef1-43f8-bcab-07ef9b7ea556 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:47:13.996862 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-222cb88a-cef1-43f8-bcab-07ef9b7ea556 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:47:13.998340 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-222cb88a-cef1-43f8-bcab-07ef9b7ea556 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:47:13.999570 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-222cb88a-cef1-43f8-bcab-07ef9b7ea556 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] 10.222.0.26 "GET /compute/v2.1/servers/85efeeae-0648-4a33-bc74-c0f0d25880aa" status: 200 len: 1717 microversion: 2.20 time: 0.215420 Oct 19 00:47:14.000143 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 623/738] 10.222.0.26 () {60 vars in 1277 bytes} [Wed Oct 19 00:47:13 2022] GET /compute/v2.1/servers/85efeeae-0648-4a33-bc74-c0f0d25880aa => generated 1717 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 19 00:47:14.178545 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-7c82cb33-60b0-4248-b111-fae031301321 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:47:14.179071 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-7c82cb33-60b0-4248-b111-fae031301321 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] [instance: 0b7c19ee-6d89-4855-bf4b-d35b65bb170b] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:47:14.185813 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-7c82cb33-60b0-4248-b111-fae031301321 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:47:14.186213 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-7c82cb33-60b0-4248-b111-fae031301321 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:47:14.186706 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-7c82cb33-60b0-4248-b111-fae031301321 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:47:14.187436 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-7c82cb33-60b0-4248-b111-fae031301321 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:47:14.187892 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-7c82cb33-60b0-4248-b111-fae031301321 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:47:14.188317 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-7c82cb33-60b0-4248-b111-fae031301321 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:47:14.313678 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-7c82cb33-60b0-4248-b111-fae031301321 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f738e67d8c2340028da1254b60d7810e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8684504b0eff415b9063cebae21ca667', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:47:14.335751 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-7c82cb33-60b0-4248-b111-fae031301321 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] 10.222.0.26 "GET /compute/v2.1/servers/0b7c19ee-6d89-4855-bf4b-d35b65bb170b" status: 200 len: 1509 microversion: 2.1 time: 0.161794 Oct 19 00:47:14.336752 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 624/739] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:47:14 2022] GET /compute/v2.1/servers/0b7c19ee-6d89-4855-bf4b-d35b65bb170b => generated 1509 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 19 00:47:14.369723 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [req-263ece15-7d32-432f-b7e6-ea626f61bf57 req-2cc39f98-fb18-4bc7-96aa-9c06b1c6198d service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "0b7c19ee-6d89-4855-bf4b-d35b65bb170b", "name": "network-vif-deleted", "tag": "3f92cbb3-bb62-457d-b0d7-92947884c5fb"}]} {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 19 00:47:14.386321 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-263ece15-7d32-432f-b7e6-ea626f61bf57 req-2cc39f98-fb18-4bc7-96aa-9c06b1c6198d service nova] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:47:14.387414 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-263ece15-7d32-432f-b7e6-ea626f61bf57 req-2cc39f98-fb18-4bc7-96aa-9c06b1c6198d service nova] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:47:14.388195 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-263ece15-7d32-432f-b7e6-ea626f61bf57 req-2cc39f98-fb18-4bc7-96aa-9c06b1c6198d service nova] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:47:14.413934 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.compute.server_external_events [req-263ece15-7d32-432f-b7e6-ea626f61bf57 req-2cc39f98-fb18-4bc7-96aa-9c06b1c6198d service nova] Creating event network-vif-deleted:3f92cbb3-bb62-457d-b0d7-92947884c5fb for instance 0b7c19ee-6d89-4855-bf4b-d35b65bb170b on hv-cinder-maste Oct 19 00:47:14.424739 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [req-263ece15-7d32-432f-b7e6-ea626f61bf57 req-2cc39f98-fb18-4bc7-96aa-9c06b1c6198d service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.079748 Oct 19 00:47:14.436280 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 625/740] 10.222.0.26 () {64 vars in 1317 bytes} [Wed Oct 19 00:47:14 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 19 00:47:15.372776 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-3db98416-acc0-46e7-a062-19878ed6a32e tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:47:15.373729 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-3db98416-acc0-46e7-a062-19878ed6a32e tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] [instance: 0b7c19ee-6d89-4855-bf4b-d35b65bb170b] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:47:15.382242 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-3db98416-acc0-46e7-a062-19878ed6a32e tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:47:15.383045 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-3db98416-acc0-46e7-a062-19878ed6a32e tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:47:15.383488 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-3db98416-acc0-46e7-a062-19878ed6a32e tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:47:15.384200 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-3db98416-acc0-46e7-a062-19878ed6a32e tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:47:15.384844 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-3db98416-acc0-46e7-a062-19878ed6a32e tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:47:15.385276 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-3db98416-acc0-46e7-a062-19878ed6a32e tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:47:15.424401 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-3db98416-acc0-46e7-a062-19878ed6a32e tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:47:15.425006 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-3db98416-acc0-46e7-a062-19878ed6a32e tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:47:15.425453 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-3db98416-acc0-46e7-a062-19878ed6a32e tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:47:15.425963 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.objects.instance [None req-3db98416-acc0-46e7-a062-19878ed6a32e tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lazy-loading 'fault' on Instance uuid 0b7c19ee-6d89-4855-bf4b-d35b65bb170b {{(pid=125641) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 19 00:47:15.507765 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-3db98416-acc0-46e7-a062-19878ed6a32e tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f738e67d8c2340028da1254b60d7810e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8684504b0eff415b9063cebae21ca667', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:47:15.523127 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-3db98416-acc0-46e7-a062-19878ed6a32e tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] 10.222.0.26 "GET /compute/v2.1/servers/0b7c19ee-6d89-4855-bf4b-d35b65bb170b" status: 200 len: 1230 microversion: 2.1 time: 0.154932 Oct 19 00:47:15.531215 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 626/741] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:47:15 2022] GET /compute/v2.1/servers/0b7c19ee-6d89-4855-bf4b-d35b65bb170b => generated 1230 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 19 00:47:15.532921 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-09d91be6-c4eb-4d99-8663-f939db659fa2 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:47:15.533900 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-09d91be6-c4eb-4d99-8663-f939db659fa2 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] [instance: 0da74d88-6672-4df0-b665-b9eec2d073c5] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:47:15.546679 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-09d91be6-c4eb-4d99-8663-f939db659fa2 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:47:15.547874 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-09d91be6-c4eb-4d99-8663-f939db659fa2 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:47:15.548850 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-09d91be6-c4eb-4d99-8663-f939db659fa2 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:47:15.555252 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-09d91be6-c4eb-4d99-8663-f939db659fa2 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:47:15.556611 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-09d91be6-c4eb-4d99-8663-f939db659fa2 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:47:15.557673 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-09d91be6-c4eb-4d99-8663-f939db659fa2 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:47:15.603894 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-09d91be6-c4eb-4d99-8663-f939db659fa2 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] 10.222.0.26 "GET /compute/v2.1/servers/0da74d88-6672-4df0-b665-b9eec2d073c5/ips" status: 200 len: 148 microversion: 2.1 time: 0.078022 Oct 19 00:47:15.605284 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 627/742] 10.222.0.26 () {58 vars in 1248 bytes} [Wed Oct 19 00:47:15 2022] GET /compute/v2.1/servers/0da74d88-6672-4df0-b665-b9eec2d073c5/ips => generated 148 bytes in 80 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 19 00:47:16.565112 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-c57ef44d-cf98-4fe6-b0a8-51741be9334b tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:47:16.566267 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-c57ef44d-cf98-4fe6-b0a8-51741be9334b tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] [instance: 0b7c19ee-6d89-4855-bf4b-d35b65bb170b] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:47:16.577233 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-c57ef44d-cf98-4fe6-b0a8-51741be9334b tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:47:16.578073 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-c57ef44d-cf98-4fe6-b0a8-51741be9334b tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:47:16.578882 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-c57ef44d-cf98-4fe6-b0a8-51741be9334b tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:47:16.580122 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-c57ef44d-cf98-4fe6-b0a8-51741be9334b tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:47:16.580913 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-c57ef44d-cf98-4fe6-b0a8-51741be9334b tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:47:16.581665 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-c57ef44d-cf98-4fe6-b0a8-51741be9334b tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:47:16.608521 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.wsgi [None req-c57ef44d-cf98-4fe6-b0a8-51741be9334b tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] HTTP exception thrown: Instance 0b7c19ee-6d89-4855-bf4b-d35b65bb170b could not be found. Oct 19 00:47:16.611636 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-c57ef44d-cf98-4fe6-b0a8-51741be9334b tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Request method failure captured: Oct 19 00:47:16.611636 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: request: GET /compute/v2.1/servers/0b7c19ee-6d89-4855-bf4b-d35b65bb170b HTTP/1.1 Oct 19 00:47:16.611636 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Accept: application/json Oct 19 00:47:16.611636 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Accept-Encoding: identity Oct 19 00:47:16.611636 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Connection: close Oct 19 00:47:16.611636 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Content-Length: 0 Oct 19 00:47:16.611636 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Content-Type: application/json Oct 19 00:47:16.611636 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Host: 10.222.0.26 Oct 19 00:47:16.611636 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Openstack-System-Scope: None Oct 19 00:47:16.611636 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: User-Agent: python-urllib3/1.26.12 Oct 19 00:47:16.611636 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Auth-Token: gAAAAABjT0hR4IKMV3eYr4-XgwodnpQA_lLDwRuR0LL79RvbIfnBA86hBsDcYLHHfVVxdm7zIXuLtp8bh_RiiYg07mmQQk7BvVbyFL3sDfn8L39RJ-FrnzBE24hbvJuW7M5tKiYBZDCP4wGZP3byE26APASrrTWCPUZIjyM-NO7IKZawq3TATyw Oct 19 00:47:16.611636 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Domain-Id: None Oct 19 00:47:16.611636 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Domain-Name: None Oct 19 00:47:16.611636 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Identity-Status: Confirmed Oct 19 00:47:16.611636 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Is-Admin-Project: True Oct 19 00:47:16.611636 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Project-Domain-Id: default Oct 19 00:47:16.611636 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Project-Domain-Name: Default Oct 19 00:47:16.611636 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Project-Id: 8684504b0eff415b9063cebae21ca667 Oct 19 00:47:16.611636 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Project-Name: tempest-TestVolumeBootPattern-1327038966 Oct 19 00:47:16.611636 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Role: member,reader Oct 19 00:47:16.611636 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Roles: member,reader Oct 19 00:47:16.613297 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/8684504b0eff415b9063cebae21ca667"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/8684504b0eff415b9063cebae21ca667"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/8684504b0eff415b9063cebae21ca667"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_8684504b0eff415b9063cebae21ca667"}]}] Oct 19 00:47:16.613297 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Tenant: tempest-TestVolumeBootPattern-1327038966 Oct 19 00:47:16.613297 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Tenant-Id: 8684504b0eff415b9063cebae21ca667 Oct 19 00:47:16.613297 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Tenant-Name: tempest-TestVolumeBootPattern-1327038966 Oct 19 00:47:16.613297 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User: tempest-TestVolumeBootPattern-1327038966-project Oct 19 00:47:16.613297 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User-Domain-Id: default Oct 19 00:47:16.613297 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User-Domain-Name: Default Oct 19 00:47:16.613297 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User-Id: f738e67d8c2340028da1254b60d7810e Oct 19 00:47:16.613297 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User-Name: tempest-TestVolumeBootPattern-1327038966-project Oct 19 00:47:16.613297 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: method: > Oct 19 00:47:16.613297 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: exception: Instance 0b7c19ee-6d89-4855-bf4b-d35b65bb170b could not be found. Oct 19 00:47:16.613297 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: action_args: {'id': '0b7c19ee-6d89-4855-bf4b-d35b65bb170b'} Oct 19 00:47:16.614249 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 19 00:47:16.614249 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 19 00:47:16.614249 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 19 00:47:16.614249 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 19 00:47:16.614249 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 19 00:47:16.614249 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 19 00:47:16.614249 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 19 00:47:16.614249 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 19 00:47:16.614249 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 19 00:47:16.614249 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 19 00:47:16.614249 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 0b7c19ee-6d89-4855-bf4b-d35b65bb170b could not be found. Oct 19 00:47:16.614249 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:47:16.614249 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 19 00:47:16.614249 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:47:16.614249 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 19 00:47:16.614249 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 19 00:47:16.614249 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 19 00:47:16.614249 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 19 00:47:16.614249 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 19 00:47:16.614249 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 19 00:47:16.615586 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 19 00:47:16.615586 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 19 00:47:16.615586 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 19 00:47:16.615586 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 19 00:47:16.615586 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 19 00:47:16.615586 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 19 00:47:16.615586 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 19 00:47:16.615586 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 0b7c19ee-6d89-4855-bf4b-d35b65bb170b could not be found. Oct 19 00:47:16.615586 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:47:16.615586 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 19 00:47:16.615586 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:47:16.615586 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 19 00:47:16.615586 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 19 00:47:16.615586 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 19 00:47:16.615586 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 19 00:47:16.615586 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 19 00:47:16.615586 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 0b7c19ee-6d89-4855-bf4b-d35b65bb170b could not be found. Oct 19 00:47:16.615586 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:47:16.616711 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-c57ef44d-cf98-4fe6-b0a8-51741be9334b tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Returning 404 to user: Instance 0b7c19ee-6d89-4855-bf4b-d35b65bb170b could not be found. {{(pid=125641) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 19 00:47:16.616711 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-c57ef44d-cf98-4fe6-b0a8-51741be9334b tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] 10.222.0.26 "GET /compute/v2.1/servers/0b7c19ee-6d89-4855-bf4b-d35b65bb170b" status: 404 len: 111 microversion: 2.1 time: 0.056315 Oct 19 00:47:16.616711 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 628/743] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:47:16 2022] GET /compute/v2.1/servers/0b7c19ee-6d89-4855-bf4b-d35b65bb170b => generated 111 bytes in 59 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 19 00:47:16.740454 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-66f7fecd-52a0-46c2-ba59-6c249c85f305 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestVolumeBootPattern-server-1266987111", "imageRef": "", "flavorRef": "452", "key_name": "tempest-TestVolumeBootPattern-1204843348", "security_groups": [{"name": "tempest-secgroup-smoke-1056832705"}], "block_device_mapping_v2": [{"uuid": "c5018c37-69d3-44f9-ac3c-7a8377379ade", "source_type": "volume", "destination_type": "volume", "boot_index": 0, "delete_on_termination": false}], "networks": [{"uuid": "cfe47441-dd27-4b7c-958a-cc2e5c8243ac"}]}} {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 19 00:47:16.875144 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.network.neutron [None req-66f7fecd-52a0-46c2-ba59-6c249c85f305 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] validate_networks() for [('cfe47441-dd27-4b7c-958a-cc2e5c8243ac', None, None, None, None, None)] {{(pid=125641) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 19 00:47:17.320294 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.virt.hardware [None req-66f7fecd-52a0-46c2-ba59-6c249c85f305 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Flavor limits 0:0:0 {{(pid=125641) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 19 00:47:17.320788 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.virt.hardware [None req-66f7fecd-52a0-46c2-ba59-6c249c85f305 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Image limits 0:0:0 {{(pid=125641) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 19 00:47:17.321209 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.virt.hardware [None req-66f7fecd-52a0-46c2-ba59-6c249c85f305 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Flavor pref 0:0:0 {{(pid=125641) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 19 00:47:17.321615 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.virt.hardware [None req-66f7fecd-52a0-46c2-ba59-6c249c85f305 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Image pref 0:0:0 {{(pid=125641) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 19 00:47:17.322025 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.virt.hardware [None req-66f7fecd-52a0-46c2-ba59-6c249c85f305 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=125641) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 19 00:47:17.332759 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-66f7fecd-52a0-46c2-ba59-6c249c85f305 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:47:17.333290 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-66f7fecd-52a0-46c2-ba59-6c249c85f305 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:47:17.333828 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-66f7fecd-52a0-46c2-ba59-6c249c85f305 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:47:17.334468 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-66f7fecd-52a0-46c2-ba59-6c249c85f305 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:47:17.334957 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-66f7fecd-52a0-46c2-ba59-6c249c85f305 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:47:17.335382 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-66f7fecd-52a0-46c2-ba59-6c249c85f305 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:47:17.355927 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.quota [None req-66f7fecd-52a0-46c2-ba59-6c249c85f305 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Getting quotas for project 8684504b0eff415b9063cebae21ca667. Resources: {'cores', 'ram', 'instances'} {{(pid=125641) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 19 00:47:17.360884 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.quota [None req-66f7fecd-52a0-46c2-ba59-6c249c85f305 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Getting quotas for user f738e67d8c2340028da1254b60d7810e and project 8684504b0eff415b9063cebae21ca667. Resources: {'cores', 'ram', 'instances'} {{(pid=125641) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 19 00:47:17.430615 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-66f7fecd-52a0-46c2-ba59-6c249c85f305 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Going to run 1 instances... {{(pid=125641) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 19 00:47:17.434557 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-66f7fecd-52a0-46c2-ba59-6c249c85f305 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] [instance: d2b07e55-0b24-4055-889f-e69cdf22f251] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=False,deleted=,deleted_at=,destination_type='volume',device_name=None,device_type=None,disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id=None,instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='volume',tag=None,updated_at=,uuid=,volume_id='c5018c37-69d3-44f9-ac3c-7a8377379ade',volume_size=None,volume_type=None)] {{(pid=125641) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 19 00:47:17.596133 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-66f7fecd-52a0-46c2-ba59-6c249c85f305 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 202 len: 406 microversion: 2.1 time: 0.859186 Oct 19 00:47:17.597189 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 629/744] 10.222.0.26 () {60 vars in 1147 bytes} [Wed Oct 19 00:47:16 2022] POST /compute/v2.1/servers => generated 406 bytes in 862 msecs (HTTP/1.1 202) 10 headers in 451 bytes (2 switches on core 0) Oct 19 00:47:17.605077 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-b1127647-4bc7-4d01-a7be-e8a582b90a8b tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "5e352941-289d-481f-8a04-7b63894d0d74"}} {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 19 00:47:17.608436 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-b1127647-4bc7-4d01-a7be-e8a582b90a8b tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] [instance: 0da74d88-6672-4df0-b665-b9eec2d073c5] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:47:17.616799 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-b1127647-4bc7-4d01-a7be-e8a582b90a8b tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:47:17.617641 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-b1127647-4bc7-4d01-a7be-e8a582b90a8b tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:47:17.619191 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-b1127647-4bc7-4d01-a7be-e8a582b90a8b tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:47:17.620094 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-b1127647-4bc7-4d01-a7be-e8a582b90a8b tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:47:17.620669 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-b1127647-4bc7-4d01-a7be-e8a582b90a8b tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:47:17.621199 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-b1127647-4bc7-4d01-a7be-e8a582b90a8b tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:47:17.993177 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-b1127647-4bc7-4d01-a7be-e8a582b90a8b tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] 10.222.0.26 "POST /compute/v2.1/servers/0da74d88-6672-4df0-b665-b9eec2d073c5/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.393107 Oct 19 00:47:17.993985 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 630/745] 10.222.0.26 () {60 vars in 1323 bytes} [Wed Oct 19 00:47:17 2022] POST /compute/v2.1/servers/0da74d88-6672-4df0-b665-b9eec2d073c5/os-volume_attachments => generated 194 bytes in 395 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 19 00:47:18.001500 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-5f5b061a-3369-4bfb-bdf3-3a4132ed6231 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:47:18.003070 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-5f5b061a-3369-4bfb-bdf3-3a4132ed6231 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] [instance: d2b07e55-0b24-4055-889f-e69cdf22f251] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:47:18.099078 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-5f5b061a-3369-4bfb-bdf3-3a4132ed6231 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f738e67d8c2340028da1254b60d7810e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8684504b0eff415b9063cebae21ca667', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:47:18.109397 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-5f5b061a-3369-4bfb-bdf3-3a4132ed6231 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] 10.222.0.26 "GET /compute/v2.1/servers/d2b07e55-0b24-4055-889f-e69cdf22f251" status: 200 len: 1095 microversion: 2.1 time: 0.113492 Oct 19 00:47:18.109871 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 631/746] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:47:17 2022] GET /compute/v2.1/servers/d2b07e55-0b24-4055-889f-e69cdf22f251 => generated 1095 bytes in 115 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:47:19.148429 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-1d8ece71-ab33-4366-a532-caad986a0c01 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:47:19.151572 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-1d8ece71-ab33-4366-a532-caad986a0c01 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] [instance: d2b07e55-0b24-4055-889f-e69cdf22f251] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:47:19.173366 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-1d8ece71-ab33-4366-a532-caad986a0c01 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:47:19.174334 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-1d8ece71-ab33-4366-a532-caad986a0c01 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:47:19.175402 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-1d8ece71-ab33-4366-a532-caad986a0c01 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:47:19.178307 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-1d8ece71-ab33-4366-a532-caad986a0c01 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:47:19.179657 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-1d8ece71-ab33-4366-a532-caad986a0c01 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:47:19.180379 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-1d8ece71-ab33-4366-a532-caad986a0c01 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:47:19.297619 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-1d8ece71-ab33-4366-a532-caad986a0c01 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f738e67d8c2340028da1254b60d7810e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8684504b0eff415b9063cebae21ca667', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:47:19.312509 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-1d8ece71-ab33-4366-a532-caad986a0c01 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] 10.222.0.26 "GET /compute/v2.1/servers/d2b07e55-0b24-4055-889f-e69cdf22f251" status: 200 len: 1211 microversion: 2.1 time: 0.169542 Oct 19 00:47:19.314135 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 632/747] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:47:19 2022] GET /compute/v2.1/servers/d2b07e55-0b24-4055-889f-e69cdf22f251 => generated 1211 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 19 00:47:20.354599 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-afe94c35-c485-445b-b605-b6c001a0ebf2 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:47:20.355666 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-afe94c35-c485-445b-b605-b6c001a0ebf2 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] [instance: d2b07e55-0b24-4055-889f-e69cdf22f251] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:47:20.370454 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-afe94c35-c485-445b-b605-b6c001a0ebf2 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:47:20.372095 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-afe94c35-c485-445b-b605-b6c001a0ebf2 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:47:20.373120 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-afe94c35-c485-445b-b605-b6c001a0ebf2 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:47:20.374632 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-afe94c35-c485-445b-b605-b6c001a0ebf2 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:47:20.375812 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-afe94c35-c485-445b-b605-b6c001a0ebf2 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:47:20.376801 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-afe94c35-c485-445b-b605-b6c001a0ebf2 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:47:20.623814 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-afe94c35-c485-445b-b605-b6c001a0ebf2 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f738e67d8c2340028da1254b60d7810e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8684504b0eff415b9063cebae21ca667', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:47:20.635361 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-afe94c35-c485-445b-b605-b6c001a0ebf2 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] 10.222.0.26 "GET /compute/v2.1/servers/d2b07e55-0b24-4055-889f-e69cdf22f251" status: 200 len: 1279 microversion: 2.1 time: 0.287766 Oct 19 00:47:20.636910 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 633/748] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:47:20 2022] GET /compute/v2.1/servers/d2b07e55-0b24-4055-889f-e69cdf22f251 => generated 1279 bytes in 291 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 19 00:47:20.661967 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Oct 19 00:47:20.662640 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: Wed Oct 19 00:47:20 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers (ip 10.222.0.26) !!! Oct 19 00:47:20.893816 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [req-ba4d53b2-03f6-4a0e-86e5-b2de3dd99bfb req-00ce4cdd-69d5-4215-b586-b7ab365cff85 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "d2b07e55-0b24-4055-889f-e69cdf22f251", "tag": "94f720c6-9b61-4983-ab49-77c9cdd0fe18"}]} {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 19 00:47:20.913182 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-ba4d53b2-03f6-4a0e-86e5-b2de3dd99bfb req-00ce4cdd-69d5-4215-b586-b7ab365cff85 service nova] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:47:20.914150 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-ba4d53b2-03f6-4a0e-86e5-b2de3dd99bfb req-00ce4cdd-69d5-4215-b586-b7ab365cff85 service nova] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:47:20.914983 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-ba4d53b2-03f6-4a0e-86e5-b2de3dd99bfb req-00ce4cdd-69d5-4215-b586-b7ab365cff85 service nova] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:47:20.938392 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.compute.server_external_events [req-ba4d53b2-03f6-4a0e-86e5-b2de3dd99bfb req-00ce4cdd-69d5-4215-b586-b7ab365cff85 service nova] Creating event network-changed:94f720c6-9b61-4983-ab49-77c9cdd0fe18 for instance d2b07e55-0b24-4055-889f-e69cdf22f251 on hv-cinder-maste Oct 19 00:47:20.949542 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [req-ba4d53b2-03f6-4a0e-86e5-b2de3dd99bfb req-00ce4cdd-69d5-4215-b586-b7ab365cff85 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.061144 Oct 19 00:47:20.950647 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 634/749] 10.222.0.26 () {64 vars in 1317 bytes} [Wed Oct 19 00:47:20 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 64 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 19 00:47:21.679838 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-b7e33dd9-17bc-4ad5-835e-353c09cdbce8 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:47:21.682473 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-b7e33dd9-17bc-4ad5-835e-353c09cdbce8 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] [instance: d2b07e55-0b24-4055-889f-e69cdf22f251] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:47:21.690751 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-b7e33dd9-17bc-4ad5-835e-353c09cdbce8 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:47:21.691292 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-b7e33dd9-17bc-4ad5-835e-353c09cdbce8 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:47:21.692538 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-b7e33dd9-17bc-4ad5-835e-353c09cdbce8 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:47:21.693409 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-b7e33dd9-17bc-4ad5-835e-353c09cdbce8 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:47:21.693904 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-b7e33dd9-17bc-4ad5-835e-353c09cdbce8 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:47:21.694341 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-b7e33dd9-17bc-4ad5-835e-353c09cdbce8 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:47:21.914556 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-b7e33dd9-17bc-4ad5-835e-353c09cdbce8 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f738e67d8c2340028da1254b60d7810e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8684504b0eff415b9063cebae21ca667', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:47:21.924662 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-b7e33dd9-17bc-4ad5-835e-353c09cdbce8 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] 10.222.0.26 "GET /compute/v2.1/servers/d2b07e55-0b24-4055-889f-e69cdf22f251" status: 200 len: 1279 microversion: 2.1 time: 0.253433 Oct 19 00:47:21.925538 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 635/750] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:47:21 2022] GET /compute/v2.1/servers/d2b07e55-0b24-4055-889f-e69cdf22f251 => generated 1279 bytes in 256 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 19 00:47:22.374870 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-2bfa9726-c910-4598-b0ae-e7b45d9a4b06 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:47:22.375940 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-2bfa9726-c910-4598-b0ae-e7b45d9a4b06 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] [instance: b38d4219-8912-4fd9-9d02-7848016f7954] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:47:22.387193 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-2bfa9726-c910-4598-b0ae-e7b45d9a4b06 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:47:22.388042 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-2bfa9726-c910-4598-b0ae-e7b45d9a4b06 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:47:22.388804 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-2bfa9726-c910-4598-b0ae-e7b45d9a4b06 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:47:22.390050 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-2bfa9726-c910-4598-b0ae-e7b45d9a4b06 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:47:22.390885 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-2bfa9726-c910-4598-b0ae-e7b45d9a4b06 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:47:22.391638 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-2bfa9726-c910-4598-b0ae-e7b45d9a4b06 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:47:22.439824 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-2bfa9726-c910-4598-b0ae-e7b45d9a4b06 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] 10.222.0.26 "GET /compute/v2.1/servers/b38d4219-8912-4fd9-9d02-7848016f7954/ips" status: 200 len: 146 microversion: 2.1 time: 0.072068 Oct 19 00:47:22.440837 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 636/751] 10.222.0.26 () {58 vars in 1248 bytes} [Wed Oct 19 00:47:22 2022] GET /compute/v2.1/servers/b38d4219-8912-4fd9-9d02-7848016f7954/ips => generated 146 bytes in 75 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 19 00:47:22.949900 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-bcc84963-aa2d-4c06-a6b7-15dba2e6e7ef tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:47:22.950775 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-bcc84963-aa2d-4c06-a6b7-15dba2e6e7ef tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] [instance: d2b07e55-0b24-4055-889f-e69cdf22f251] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:47:22.959777 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-bcc84963-aa2d-4c06-a6b7-15dba2e6e7ef tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:47:22.960339 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-bcc84963-aa2d-4c06-a6b7-15dba2e6e7ef tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:47:22.961099 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-bcc84963-aa2d-4c06-a6b7-15dba2e6e7ef tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:47:22.961941 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-bcc84963-aa2d-4c06-a6b7-15dba2e6e7ef tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:47:22.962577 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-bcc84963-aa2d-4c06-a6b7-15dba2e6e7ef tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:47:22.963200 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-bcc84963-aa2d-4c06-a6b7-15dba2e6e7ef tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:47:23.122657 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-bcc84963-aa2d-4c06-a6b7-15dba2e6e7ef tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f738e67d8c2340028da1254b60d7810e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8684504b0eff415b9063cebae21ca667', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:47:23.132904 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-bcc84963-aa2d-4c06-a6b7-15dba2e6e7ef tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] 10.222.0.26 "GET /compute/v2.1/servers/d2b07e55-0b24-4055-889f-e69cdf22f251" status: 200 len: 1279 microversion: 2.1 time: 0.187134 Oct 19 00:47:23.133596 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 637/752] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:47:22 2022] GET /compute/v2.1/servers/d2b07e55-0b24-4055-889f-e69cdf22f251 => generated 1279 bytes in 189 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 19 00:47:24.157834 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-6895139b-f479-495a-9eba-9c35057d239d tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:47:24.158586 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-6895139b-f479-495a-9eba-9c35057d239d tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] [instance: d2b07e55-0b24-4055-889f-e69cdf22f251] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:47:24.168003 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-6895139b-f479-495a-9eba-9c35057d239d tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:47:24.168577 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-6895139b-f479-495a-9eba-9c35057d239d tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:47:24.169077 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-6895139b-f479-495a-9eba-9c35057d239d tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:47:24.169911 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-6895139b-f479-495a-9eba-9c35057d239d tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:47:24.170581 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-6895139b-f479-495a-9eba-9c35057d239d tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:47:24.171166 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-6895139b-f479-495a-9eba-9c35057d239d tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:47:24.178923 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG dbcounter [-] [125641] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=125641) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 00:47:24.379352 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-6895139b-f479-495a-9eba-9c35057d239d tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f738e67d8c2340028da1254b60d7810e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8684504b0eff415b9063cebae21ca667', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:47:24.389241 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-6895139b-f479-495a-9eba-9c35057d239d tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] 10.222.0.26 "GET /compute/v2.1/servers/d2b07e55-0b24-4055-889f-e69cdf22f251" status: 200 len: 1279 microversion: 2.1 time: 0.235584 Oct 19 00:47:24.390144 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 638/753] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:47:24 2022] GET /compute/v2.1/servers/d2b07e55-0b24-4055-889f-e69cdf22f251 => generated 1279 bytes in 237 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:47:25.422011 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-cf11f109-12e6-4b07-9969-cd380f5f7c6d tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:47:25.423138 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-cf11f109-12e6-4b07-9969-cd380f5f7c6d tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] [instance: d2b07e55-0b24-4055-889f-e69cdf22f251] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:47:25.436006 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-cf11f109-12e6-4b07-9969-cd380f5f7c6d tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:47:25.436880 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-cf11f109-12e6-4b07-9969-cd380f5f7c6d tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:47:25.437676 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-cf11f109-12e6-4b07-9969-cd380f5f7c6d tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:47:25.439011 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-cf11f109-12e6-4b07-9969-cd380f5f7c6d tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:47:25.439820 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-cf11f109-12e6-4b07-9969-cd380f5f7c6d tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:47:25.440552 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-cf11f109-12e6-4b07-9969-cd380f5f7c6d tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:47:25.677488 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-cf11f109-12e6-4b07-9969-cd380f5f7c6d tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f738e67d8c2340028da1254b60d7810e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8684504b0eff415b9063cebae21ca667', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:47:25.688749 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-cf11f109-12e6-4b07-9969-cd380f5f7c6d tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] 10.222.0.26 "GET /compute/v2.1/servers/d2b07e55-0b24-4055-889f-e69cdf22f251" status: 200 len: 1279 microversion: 2.1 time: 0.273833 Oct 19 00:47:25.689346 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 639/754] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:47:25 2022] GET /compute/v2.1/servers/d2b07e55-0b24-4055-889f-e69cdf22f251 => generated 1279 bytes in 276 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:47:25.694651 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-42873178-45ec-420f-b6d6-4de85110dfa1 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "68f7c32a-29e2-48c4-a946-649b0aa6acf3"}} {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 19 00:47:25.697279 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-42873178-45ec-420f-b6d6-4de85110dfa1 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] [instance: b38d4219-8912-4fd9-9d02-7848016f7954] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:47:25.703678 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-42873178-45ec-420f-b6d6-4de85110dfa1 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:47:25.704179 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-42873178-45ec-420f-b6d6-4de85110dfa1 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:47:25.704621 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-42873178-45ec-420f-b6d6-4de85110dfa1 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:47:25.705343 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-42873178-45ec-420f-b6d6-4de85110dfa1 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:47:25.705806 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-42873178-45ec-420f-b6d6-4de85110dfa1 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:47:25.706234 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-42873178-45ec-420f-b6d6-4de85110dfa1 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:47:26.375638 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-42873178-45ec-420f-b6d6-4de85110dfa1 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] 10.222.0.26 "POST /compute/v2.1/servers/b38d4219-8912-4fd9-9d02-7848016f7954/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.685034 Oct 19 00:47:26.376547 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 640/755] 10.222.0.26 () {60 vars in 1323 bytes} [Wed Oct 19 00:47:25 2022] POST /compute/v2.1/servers/b38d4219-8912-4fd9-9d02-7848016f7954/os-volume_attachments => generated 194 bytes in 687 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 19 00:47:26.383713 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-f172db4d-5fd6-49ef-86a6-a5eb66da4bd3 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:47:26.384312 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-f172db4d-5fd6-49ef-86a6-a5eb66da4bd3 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] [instance: 0da74d88-6672-4df0-b665-b9eec2d073c5] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:47:26.392288 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-f172db4d-5fd6-49ef-86a6-a5eb66da4bd3 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:47:26.392790 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-f172db4d-5fd6-49ef-86a6-a5eb66da4bd3 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:47:26.393240 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-f172db4d-5fd6-49ef-86a6-a5eb66da4bd3 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:47:26.393976 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-f172db4d-5fd6-49ef-86a6-a5eb66da4bd3 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:47:26.394565 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-f172db4d-5fd6-49ef-86a6-a5eb66da4bd3 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:47:26.394969 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-f172db4d-5fd6-49ef-86a6-a5eb66da4bd3 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:47:26.499881 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.objects.instance [None req-f172db4d-5fd6-49ef-86a6-a5eb66da4bd3 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lazy-loading 'services' on Instance uuid 0da74d88-6672-4df0-b665-b9eec2d073c5 {{(pid=125641) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 19 00:47:26.591405 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-f172db4d-5fd6-49ef-86a6-a5eb66da4bd3 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] 10.222.0.26 "DELETE /compute/v2.1/servers/0da74d88-6672-4df0-b665-b9eec2d073c5/os-volume_attachments/5e352941-289d-481f-8a04-7b63894d0d74" status: 202 len: 0 microversion: 2.1 time: 0.213317 Oct 19 00:47:26.592023 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 641/756] 10.222.0.26 () {58 vars in 1416 bytes} [Wed Oct 19 00:47:26 2022] DELETE /compute/v2.1/servers/0da74d88-6672-4df0-b665-b9eec2d073c5/os-volume_attachments/5e352941-289d-481f-8a04-7b63894d0d74 => generated 0 bytes in 215 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 19 00:47:26.602550 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-5f0f6c72-930b-4089-8d3d-04a66a7e9a18 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:47:26.605172 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-5f0f6c72-930b-4089-8d3d-04a66a7e9a18 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] [instance: 0da74d88-6672-4df0-b665-b9eec2d073c5] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:47:26.609270 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG dbcounter [-] [125641] Writing DB stats nova_api:SELECT=96,nova_api:UPDATE=1,nova_api:INSERT=3 {{(pid=125641) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 00:47:26.613936 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-5f0f6c72-930b-4089-8d3d-04a66a7e9a18 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:47:26.614434 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-5f0f6c72-930b-4089-8d3d-04a66a7e9a18 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:47:26.614904 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-5f0f6c72-930b-4089-8d3d-04a66a7e9a18 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:47:26.615636 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-5f0f6c72-930b-4089-8d3d-04a66a7e9a18 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:47:26.616094 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-5f0f6c72-930b-4089-8d3d-04a66a7e9a18 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:47:26.616524 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-5f0f6c72-930b-4089-8d3d-04a66a7e9a18 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:47:26.667667 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-5f0f6c72-930b-4089-8d3d-04a66a7e9a18 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] 10.222.0.26 "GET /compute/v2.1/servers/0da74d88-6672-4df0-b665-b9eec2d073c5/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.069992 Oct 19 00:47:26.668276 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 642/757] 10.222.0.26 () {58 vars in 1302 bytes} [Wed Oct 19 00:47:26 2022] GET /compute/v2.1/servers/0da74d88-6672-4df0-b665-b9eec2d073c5/os-volume_attachments => generated 197 bytes in 71 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 19 00:47:26.728304 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-e0b9b190-e63e-4779-b7b0-2e5374c49f9f tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:47:26.729030 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-e0b9b190-e63e-4779-b7b0-2e5374c49f9f tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] [instance: d2b07e55-0b24-4055-889f-e69cdf22f251] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:47:26.735245 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-e0b9b190-e63e-4779-b7b0-2e5374c49f9f tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:47:26.735860 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-e0b9b190-e63e-4779-b7b0-2e5374c49f9f tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:47:26.736297 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-e0b9b190-e63e-4779-b7b0-2e5374c49f9f tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:47:26.737063 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-e0b9b190-e63e-4779-b7b0-2e5374c49f9f tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:47:26.737657 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-e0b9b190-e63e-4779-b7b0-2e5374c49f9f tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:47:26.737937 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-e0b9b190-e63e-4779-b7b0-2e5374c49f9f tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:47:26.897615 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-e0b9b190-e63e-4779-b7b0-2e5374c49f9f tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f738e67d8c2340028da1254b60d7810e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8684504b0eff415b9063cebae21ca667', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:47:26.906513 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-e0b9b190-e63e-4779-b7b0-2e5374c49f9f tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] 10.222.0.26 "GET /compute/v2.1/servers/d2b07e55-0b24-4055-889f-e69cdf22f251" status: 200 len: 1267 microversion: 2.1 time: 0.182155 Oct 19 00:47:26.907176 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 643/758] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:47:26 2022] GET /compute/v2.1/servers/d2b07e55-0b24-4055-889f-e69cdf22f251 => generated 1267 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 19 00:47:27.417638 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Oct 19 00:47:27.418201 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: Wed Oct 19 00:47:27 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers (ip 10.222.0.26) !!! Oct 19 00:47:27.684249 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-37396020-48de-4896-bcd9-858c68e794a4 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:47:27.686645 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-37396020-48de-4896-bcd9-858c68e794a4 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] [instance: 0da74d88-6672-4df0-b665-b9eec2d073c5] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:47:27.689131 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG dbcounter [-] [125641] Writing DB stats nova_cell0:SELECT=2 {{(pid=125641) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 00:47:27.696382 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-37396020-48de-4896-bcd9-858c68e794a4 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:47:27.696870 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-37396020-48de-4896-bcd9-858c68e794a4 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:47:27.697305 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-37396020-48de-4896-bcd9-858c68e794a4 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:47:27.698022 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-37396020-48de-4896-bcd9-858c68e794a4 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:47:27.698476 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-37396020-48de-4896-bcd9-858c68e794a4 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:47:27.698938 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-37396020-48de-4896-bcd9-858c68e794a4 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:47:27.734106 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-37396020-48de-4896-bcd9-858c68e794a4 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] 10.222.0.26 "GET /compute/v2.1/servers/0da74d88-6672-4df0-b665-b9eec2d073c5/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.053841 Oct 19 00:47:27.734789 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 644/759] 10.222.0.26 () {58 vars in 1302 bytes} [Wed Oct 19 00:47:27 2022] GET /compute/v2.1/servers/0da74d88-6672-4df0-b665-b9eec2d073c5/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 19 00:47:27.935464 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-a109998a-eade-417c-99c7-4049174f4050 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:47:27.936559 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-a109998a-eade-417c-99c7-4049174f4050 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] [instance: d2b07e55-0b24-4055-889f-e69cdf22f251] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:47:27.949060 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-a109998a-eade-417c-99c7-4049174f4050 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:47:27.949928 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-a109998a-eade-417c-99c7-4049174f4050 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:47:27.950742 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-a109998a-eade-417c-99c7-4049174f4050 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:47:27.951999 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-a109998a-eade-417c-99c7-4049174f4050 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:47:27.952799 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-a109998a-eade-417c-99c7-4049174f4050 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:47:27.953542 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-a109998a-eade-417c-99c7-4049174f4050 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:47:28.103893 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG dbcounter [-] [125641] Writing DB stats nova_cell0:SELECT=8 {{(pid=125641) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 00:47:28.120370 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-a109998a-eade-417c-99c7-4049174f4050 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f738e67d8c2340028da1254b60d7810e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8684504b0eff415b9063cebae21ca667', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:47:28.131273 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-a109998a-eade-417c-99c7-4049174f4050 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] 10.222.0.26 "GET /compute/v2.1/servers/d2b07e55-0b24-4055-889f-e69cdf22f251" status: 200 len: 1267 microversion: 2.1 time: 0.203266 Oct 19 00:47:28.131956 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 645/760] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:47:27 2022] GET /compute/v2.1/servers/d2b07e55-0b24-4055-889f-e69cdf22f251 => generated 1267 bytes in 205 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:47:28.748403 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-e2b24b2b-b435-4d7a-ad62-8dff81765f54 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:47:28.750820 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-e2b24b2b-b435-4d7a-ad62-8dff81765f54 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] [instance: 0da74d88-6672-4df0-b665-b9eec2d073c5] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:47:28.758746 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-e2b24b2b-b435-4d7a-ad62-8dff81765f54 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:47:28.759334 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-e2b24b2b-b435-4d7a-ad62-8dff81765f54 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:47:28.759864 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-e2b24b2b-b435-4d7a-ad62-8dff81765f54 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:47:28.760609 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-e2b24b2b-b435-4d7a-ad62-8dff81765f54 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:47:28.761074 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-e2b24b2b-b435-4d7a-ad62-8dff81765f54 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:47:28.761506 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-e2b24b2b-b435-4d7a-ad62-8dff81765f54 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:47:28.797539 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-e2b24b2b-b435-4d7a-ad62-8dff81765f54 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] 10.222.0.26 "GET /compute/v2.1/servers/0da74d88-6672-4df0-b665-b9eec2d073c5/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.053407 Oct 19 00:47:28.798129 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 646/761] 10.222.0.26 () {58 vars in 1302 bytes} [Wed Oct 19 00:47:28 2022] GET /compute/v2.1/servers/0da74d88-6672-4df0-b665-b9eec2d073c5/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 19 00:47:29.155820 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-160a13cc-3f80-4403-a2a5-daf409b52009 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:47:29.156731 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-160a13cc-3f80-4403-a2a5-daf409b52009 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] [instance: d2b07e55-0b24-4055-889f-e69cdf22f251] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:47:29.168580 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-160a13cc-3f80-4403-a2a5-daf409b52009 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:47:29.169476 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-160a13cc-3f80-4403-a2a5-daf409b52009 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:47:29.170204 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-160a13cc-3f80-4403-a2a5-daf409b52009 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:47:29.171495 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-160a13cc-3f80-4403-a2a5-daf409b52009 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:47:29.172322 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-160a13cc-3f80-4403-a2a5-daf409b52009 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:47:29.173085 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-160a13cc-3f80-4403-a2a5-daf409b52009 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:47:29.339429 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-160a13cc-3f80-4403-a2a5-daf409b52009 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f738e67d8c2340028da1254b60d7810e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8684504b0eff415b9063cebae21ca667', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:47:29.349084 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-160a13cc-3f80-4403-a2a5-daf409b52009 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] 10.222.0.26 "GET /compute/v2.1/servers/d2b07e55-0b24-4055-889f-e69cdf22f251" status: 200 len: 1267 microversion: 2.1 time: 0.196702 Oct 19 00:47:29.349758 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 647/762] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:47:29 2022] GET /compute/v2.1/servers/d2b07e55-0b24-4055-889f-e69cdf22f251 => generated 1267 bytes in 199 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:47:29.811259 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-fc090fd2-0d88-4531-904b-b89837c182b5 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:47:29.813569 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-fc090fd2-0d88-4531-904b-b89837c182b5 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] [instance: 0da74d88-6672-4df0-b665-b9eec2d073c5] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:47:29.821319 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-fc090fd2-0d88-4531-904b-b89837c182b5 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:47:29.821811 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-fc090fd2-0d88-4531-904b-b89837c182b5 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:47:29.822251 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-fc090fd2-0d88-4531-904b-b89837c182b5 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:47:29.823006 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-fc090fd2-0d88-4531-904b-b89837c182b5 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:47:29.823463 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-fc090fd2-0d88-4531-904b-b89837c182b5 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:47:29.823888 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-fc090fd2-0d88-4531-904b-b89837c182b5 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:47:29.860717 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-fc090fd2-0d88-4531-904b-b89837c182b5 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] 10.222.0.26 "GET /compute/v2.1/servers/0da74d88-6672-4df0-b665-b9eec2d073c5/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.053498 Oct 19 00:47:29.861317 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 648/763] 10.222.0.26 () {58 vars in 1302 bytes} [Wed Oct 19 00:47:29 2022] GET /compute/v2.1/servers/0da74d88-6672-4df0-b665-b9eec2d073c5/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 19 00:47:30.374430 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-3bc6d9fd-2f44-4d7f-a054-0579e0d7d813 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:47:30.375086 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-3bc6d9fd-2f44-4d7f-a054-0579e0d7d813 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] [instance: d2b07e55-0b24-4055-889f-e69cdf22f251] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:47:30.383234 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-3bc6d9fd-2f44-4d7f-a054-0579e0d7d813 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:47:30.383714 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-3bc6d9fd-2f44-4d7f-a054-0579e0d7d813 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:47:30.384155 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-3bc6d9fd-2f44-4d7f-a054-0579e0d7d813 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:47:30.384873 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-3bc6d9fd-2f44-4d7f-a054-0579e0d7d813 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:47:30.385337 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-3bc6d9fd-2f44-4d7f-a054-0579e0d7d813 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:47:30.385769 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-3bc6d9fd-2f44-4d7f-a054-0579e0d7d813 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:47:30.543259 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-3bc6d9fd-2f44-4d7f-a054-0579e0d7d813 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f738e67d8c2340028da1254b60d7810e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8684504b0eff415b9063cebae21ca667', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:47:30.555567 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-3bc6d9fd-2f44-4d7f-a054-0579e0d7d813 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] 10.222.0.26 "GET /compute/v2.1/servers/d2b07e55-0b24-4055-889f-e69cdf22f251" status: 200 len: 1267 microversion: 2.1 time: 0.185030 Oct 19 00:47:30.556722 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 649/764] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:47:30 2022] GET /compute/v2.1/servers/d2b07e55-0b24-4055-889f-e69cdf22f251 => generated 1267 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 19 00:47:30.874942 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-399a5c09-f55a-4830-a35c-2fffe568ca73 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:47:30.877369 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-399a5c09-f55a-4830-a35c-2fffe568ca73 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] [instance: 0da74d88-6672-4df0-b665-b9eec2d073c5] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:47:30.885661 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-399a5c09-f55a-4830-a35c-2fffe568ca73 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:47:30.886149 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-399a5c09-f55a-4830-a35c-2fffe568ca73 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:47:30.886617 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-399a5c09-f55a-4830-a35c-2fffe568ca73 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:47:30.887346 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-399a5c09-f55a-4830-a35c-2fffe568ca73 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:47:30.887829 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-399a5c09-f55a-4830-a35c-2fffe568ca73 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:47:30.888263 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-399a5c09-f55a-4830-a35c-2fffe568ca73 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:47:30.922469 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-399a5c09-f55a-4830-a35c-2fffe568ca73 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] 10.222.0.26 "GET /compute/v2.1/servers/0da74d88-6672-4df0-b665-b9eec2d073c5/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.052144 Oct 19 00:47:30.923175 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 650/765] 10.222.0.26 () {58 vars in 1302 bytes} [Wed Oct 19 00:47:30 2022] GET /compute/v2.1/servers/0da74d88-6672-4df0-b665-b9eec2d073c5/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 19 00:47:31.584226 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-a092a443-ad9e-4682-8e45-af4c7261c981 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:47:31.585307 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-a092a443-ad9e-4682-8e45-af4c7261c981 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] [instance: d2b07e55-0b24-4055-889f-e69cdf22f251] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:47:31.595828 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-a092a443-ad9e-4682-8e45-af4c7261c981 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:47:31.596529 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-a092a443-ad9e-4682-8e45-af4c7261c981 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:47:31.597239 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-a092a443-ad9e-4682-8e45-af4c7261c981 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:47:31.598768 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-a092a443-ad9e-4682-8e45-af4c7261c981 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:47:31.599587 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-a092a443-ad9e-4682-8e45-af4c7261c981 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:47:31.600343 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-a092a443-ad9e-4682-8e45-af4c7261c981 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:47:31.757522 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-a092a443-ad9e-4682-8e45-af4c7261c981 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f738e67d8c2340028da1254b60d7810e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8684504b0eff415b9063cebae21ca667', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:47:31.765875 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-a092a443-ad9e-4682-8e45-af4c7261c981 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] 10.222.0.26 "GET /compute/v2.1/servers/d2b07e55-0b24-4055-889f-e69cdf22f251" status: 200 len: 1267 microversion: 2.1 time: 0.188097 Oct 19 00:47:31.766537 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 651/766] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:47:31 2022] GET /compute/v2.1/servers/d2b07e55-0b24-4055-889f-e69cdf22f251 => generated 1267 bytes in 190 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:47:31.936605 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-ada46f84-f40f-426f-a5ff-6b78b03f640c tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:47:31.939565 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-ada46f84-f40f-426f-a5ff-6b78b03f640c tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] [instance: 0da74d88-6672-4df0-b665-b9eec2d073c5] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:47:31.947766 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ada46f84-f40f-426f-a5ff-6b78b03f640c tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:47:31.948343 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ada46f84-f40f-426f-a5ff-6b78b03f640c tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:47:31.948876 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ada46f84-f40f-426f-a5ff-6b78b03f640c tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:47:31.949649 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ada46f84-f40f-426f-a5ff-6b78b03f640c tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:47:31.950167 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ada46f84-f40f-426f-a5ff-6b78b03f640c tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:47:31.950698 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ada46f84-f40f-426f-a5ff-6b78b03f640c tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:47:31.980313 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG dbcounter [-] [125641] Writing DB stats nova_cell1:SELECT=96,nova_cell1:UPDATE=2,nova_cell1:INSERT=2 {{(pid=125641) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 00:47:31.989279 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-ada46f84-f40f-426f-a5ff-6b78b03f640c tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] 10.222.0.26 "GET /compute/v2.1/servers/0da74d88-6672-4df0-b665-b9eec2d073c5/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.057122 Oct 19 00:47:31.989864 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 652/767] 10.222.0.26 () {58 vars in 1302 bytes} [Wed Oct 19 00:47:31 2022] GET /compute/v2.1/servers/0da74d88-6672-4df0-b665-b9eec2d073c5/os-volume_attachments => generated 197 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 19 00:47:32.790039 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-dae3b082-6331-4c27-9e1b-519f444b87fc tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:47:32.790782 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-dae3b082-6331-4c27-9e1b-519f444b87fc tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] [instance: d2b07e55-0b24-4055-889f-e69cdf22f251] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:47:32.798802 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-dae3b082-6331-4c27-9e1b-519f444b87fc tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:47:32.799294 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-dae3b082-6331-4c27-9e1b-519f444b87fc tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:47:32.799718 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-dae3b082-6331-4c27-9e1b-519f444b87fc tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:47:32.800430 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-dae3b082-6331-4c27-9e1b-519f444b87fc tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:47:32.800964 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-dae3b082-6331-4c27-9e1b-519f444b87fc tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:47:32.801288 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-dae3b082-6331-4c27-9e1b-519f444b87fc tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:47:32.968465 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-dae3b082-6331-4c27-9e1b-519f444b87fc tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f738e67d8c2340028da1254b60d7810e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8684504b0eff415b9063cebae21ca667', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:47:32.978721 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-dae3b082-6331-4c27-9e1b-519f444b87fc tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] 10.222.0.26 "GET /compute/v2.1/servers/d2b07e55-0b24-4055-889f-e69cdf22f251" status: 200 len: 1267 microversion: 2.1 time: 0.193428 Oct 19 00:47:32.979324 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 653/768] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:47:32 2022] GET /compute/v2.1/servers/d2b07e55-0b24-4055-889f-e69cdf22f251 => generated 1267 bytes in 195 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 19 00:47:33.002528 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-899ceeec-65ef-475c-ae84-4fb627d85855 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:47:33.005218 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-899ceeec-65ef-475c-ae84-4fb627d85855 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] [instance: 0da74d88-6672-4df0-b665-b9eec2d073c5] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:47:33.012933 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-899ceeec-65ef-475c-ae84-4fb627d85855 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:47:33.013482 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-899ceeec-65ef-475c-ae84-4fb627d85855 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:47:33.013926 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-899ceeec-65ef-475c-ae84-4fb627d85855 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:47:33.014867 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-899ceeec-65ef-475c-ae84-4fb627d85855 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:47:33.015348 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-899ceeec-65ef-475c-ae84-4fb627d85855 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:47:33.015809 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-899ceeec-65ef-475c-ae84-4fb627d85855 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:47:33.059403 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-899ceeec-65ef-475c-ae84-4fb627d85855 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] 10.222.0.26 "GET /compute/v2.1/servers/0da74d88-6672-4df0-b665-b9eec2d073c5/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.060245 Oct 19 00:47:33.060173 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 654/769] 10.222.0.26 () {58 vars in 1302 bytes} [Wed Oct 19 00:47:32 2022] GET /compute/v2.1/servers/0da74d88-6672-4df0-b665-b9eec2d073c5/os-volume_attachments => generated 197 bytes in 63 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 19 00:47:34.016810 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-fa2cb215-eed4-43d6-a300-4daca64d9d9d tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:47:34.017947 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-fa2cb215-eed4-43d6-a300-4daca64d9d9d tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] [instance: d2b07e55-0b24-4055-889f-e69cdf22f251] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:47:34.030339 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-fa2cb215-eed4-43d6-a300-4daca64d9d9d tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:47:34.033498 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-fa2cb215-eed4-43d6-a300-4daca64d9d9d tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:47:34.033498 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-fa2cb215-eed4-43d6-a300-4daca64d9d9d tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:47:34.034746 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-fa2cb215-eed4-43d6-a300-4daca64d9d9d tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:47:34.036426 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-fa2cb215-eed4-43d6-a300-4daca64d9d9d tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:47:34.037192 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-fa2cb215-eed4-43d6-a300-4daca64d9d9d tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:47:34.228500 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-fa2cb215-eed4-43d6-a300-4daca64d9d9d tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f738e67d8c2340028da1254b60d7810e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8684504b0eff415b9063cebae21ca667', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:47:34.243227 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-fa2cb215-eed4-43d6-a300-4daca64d9d9d tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] 10.222.0.26 "GET /compute/v2.1/servers/d2b07e55-0b24-4055-889f-e69cdf22f251" status: 200 len: 1267 microversion: 2.1 time: 0.241809 Oct 19 00:47:34.244020 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 655/770] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:47:33 2022] GET /compute/v2.1/servers/d2b07e55-0b24-4055-889f-e69cdf22f251 => generated 1267 bytes in 245 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:47:34.251030 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-bc038fa9-b68d-41d4-aba8-650a83a29b59 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:47:34.254749 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-bc038fa9-b68d-41d4-aba8-650a83a29b59 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] [instance: 0da74d88-6672-4df0-b665-b9eec2d073c5] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:47:34.264504 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-bc038fa9-b68d-41d4-aba8-650a83a29b59 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:47:34.265553 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-bc038fa9-b68d-41d4-aba8-650a83a29b59 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:47:34.266169 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-bc038fa9-b68d-41d4-aba8-650a83a29b59 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:47:34.268285 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-bc038fa9-b68d-41d4-aba8-650a83a29b59 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:47:34.268285 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-bc038fa9-b68d-41d4-aba8-650a83a29b59 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:47:34.268285 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-bc038fa9-b68d-41d4-aba8-650a83a29b59 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:47:34.321964 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-bc038fa9-b68d-41d4-aba8-650a83a29b59 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] 10.222.0.26 "GET /compute/v2.1/servers/0da74d88-6672-4df0-b665-b9eec2d073c5/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.076644 Oct 19 00:47:34.322596 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 656/771] 10.222.0.26 () {58 vars in 1302 bytes} [Wed Oct 19 00:47:34 2022] GET /compute/v2.1/servers/0da74d88-6672-4df0-b665-b9eec2d073c5/os-volume_attachments => generated 25 bytes in 78 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Oct 19 00:47:34.327435 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-523506e6-6407-40e9-9959-1d18ecc7fe0a tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Action: 'action', calling method: >, body: {"os-stop": {}} {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 19 00:47:34.334423 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-523506e6-6407-40e9-9959-1d18ecc7fe0a tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:47:34.334951 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-523506e6-6407-40e9-9959-1d18ecc7fe0a tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:47:34.335384 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-523506e6-6407-40e9-9959-1d18ecc7fe0a tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:47:34.420752 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-523506e6-6407-40e9-9959-1d18ecc7fe0a tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] [instance: b38d4219-8912-4fd9-9d02-7848016f7954] Going to try to stop instance {{(pid=125641) force_stop /opt/stack/nova/nova/compute/api.py:2762}} Oct 19 00:47:34.481141 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-523506e6-6407-40e9-9959-1d18ecc7fe0a tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] 10.222.0.26 "POST /compute/v2.1/servers/b38d4219-8912-4fd9-9d02-7848016f7954/action" status: 202 len: 0 microversion: 2.1 time: 0.157174 Oct 19 00:47:34.482022 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 657/772] 10.222.0.26 () {60 vars in 1278 bytes} [Wed Oct 19 00:47:34 2022] POST /compute/v2.1/servers/b38d4219-8912-4fd9-9d02-7848016f7954/action => generated 0 bytes in 159 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 19 00:47:34.493260 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-d12e6484-de06-4dc5-928b-0966d1c50fab tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:47:34.494082 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-d12e6484-de06-4dc5-928b-0966d1c50fab tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] [instance: 0da74d88-6672-4df0-b665-b9eec2d073c5] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:47:34.507466 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d12e6484-de06-4dc5-928b-0966d1c50fab tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:47:34.508161 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d12e6484-de06-4dc5-928b-0966d1c50fab tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:47:34.508794 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d12e6484-de06-4dc5-928b-0966d1c50fab tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:47:34.509795 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d12e6484-de06-4dc5-928b-0966d1c50fab tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:47:34.510436 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d12e6484-de06-4dc5-928b-0966d1c50fab tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:47:34.511210 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d12e6484-de06-4dc5-928b-0966d1c50fab tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:47:34.548358 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-d12e6484-de06-4dc5-928b-0966d1c50fab tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] [instance: 0da74d88-6672-4df0-b665-b9eec2d073c5] Going to try to terminate instance {{(pid=125641) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 19 00:47:34.653070 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-d12e6484-de06-4dc5-928b-0966d1c50fab tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] 10.222.0.26 "DELETE /compute/v2.1/servers/0da74d88-6672-4df0-b665-b9eec2d073c5" status: 204 len: 0 microversion: 2.1 time: 0.169491 Oct 19 00:47:34.654026 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 658/773] 10.222.0.26 () {58 vars in 1239 bytes} [Wed Oct 19 00:47:34 2022] DELETE /compute/v2.1/servers/0da74d88-6672-4df0-b665-b9eec2d073c5 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 19 00:47:34.663949 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-7f44c30c-ab62-4a62-9068-8620bdbbea7f tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:47:34.664716 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-7f44c30c-ab62-4a62-9068-8620bdbbea7f tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] [instance: b38d4219-8912-4fd9-9d02-7848016f7954] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:47:34.671572 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-7f44c30c-ab62-4a62-9068-8620bdbbea7f tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:47:34.672064 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-7f44c30c-ab62-4a62-9068-8620bdbbea7f tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:47:34.672505 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-7f44c30c-ab62-4a62-9068-8620bdbbea7f tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:47:34.673222 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-7f44c30c-ab62-4a62-9068-8620bdbbea7f tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:47:34.673680 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-7f44c30c-ab62-4a62-9068-8620bdbbea7f tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:47:34.674111 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-7f44c30c-ab62-4a62-9068-8620bdbbea7f tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:47:34.837733 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-7f44c30c-ab62-4a62-9068-8620bdbbea7f tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a48ed2d9ded741ca91afc1fbbd9c608f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '82d7723ef95b4c3faeac9835eeb233a5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:47:34.854316 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-7f44c30c-ab62-4a62-9068-8620bdbbea7f tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] 10.222.0.26 "GET /compute/v2.1/servers/b38d4219-8912-4fd9-9d02-7848016f7954" status: 200 len: 1725 microversion: 2.1 time: 0.197874 Oct 19 00:47:34.855496 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 659/774] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:47:34 2022] GET /compute/v2.1/servers/b38d4219-8912-4fd9-9d02-7848016f7954 => generated 1725 bytes in 201 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:47:34.863987 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-a47b8ace-095f-46d7-b6e0-a8db4c049706 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:47:34.865056 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-a47b8ace-095f-46d7-b6e0-a8db4c049706 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] [instance: 0da74d88-6672-4df0-b665-b9eec2d073c5] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:47:34.875821 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-a47b8ace-095f-46d7-b6e0-a8db4c049706 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:47:34.876645 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-a47b8ace-095f-46d7-b6e0-a8db4c049706 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:47:34.877429 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-a47b8ace-095f-46d7-b6e0-a8db4c049706 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:47:34.878731 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-a47b8ace-095f-46d7-b6e0-a8db4c049706 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:47:34.879994 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-a47b8ace-095f-46d7-b6e0-a8db4c049706 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:47:34.880749 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-a47b8ace-095f-46d7-b6e0-a8db4c049706 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:47:35.057659 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-a47b8ace-095f-46d7-b6e0-a8db4c049706 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f8725c4d05b4dd79b05b43fba15980b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0b35c2038f6b4d0a9012b004f1416c00', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:47:35.068418 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-a47b8ace-095f-46d7-b6e0-a8db4c049706 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] 10.222.0.26 "GET /compute/v2.1/servers/0da74d88-6672-4df0-b665-b9eec2d073c5" status: 200 len: 1680 microversion: 2.1 time: 0.211407 Oct 19 00:47:35.069243 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 660/775] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:47:34 2022] GET /compute/v2.1/servers/0da74d88-6672-4df0-b665-b9eec2d073c5 => generated 1680 bytes in 213 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:47:35.274726 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-ffcdff0e-4d03-4842-a690-c4b497632c24 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:47:35.275607 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-ffcdff0e-4d03-4842-a690-c4b497632c24 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] [instance: d2b07e55-0b24-4055-889f-e69cdf22f251] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:47:35.286784 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ffcdff0e-4d03-4842-a690-c4b497632c24 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:47:35.287608 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ffcdff0e-4d03-4842-a690-c4b497632c24 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:47:35.288410 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ffcdff0e-4d03-4842-a690-c4b497632c24 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:47:35.289523 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ffcdff0e-4d03-4842-a690-c4b497632c24 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:47:35.290244 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ffcdff0e-4d03-4842-a690-c4b497632c24 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:47:35.290879 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ffcdff0e-4d03-4842-a690-c4b497632c24 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:47:35.501738 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-ffcdff0e-4d03-4842-a690-c4b497632c24 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f738e67d8c2340028da1254b60d7810e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8684504b0eff415b9063cebae21ca667', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:47:35.510959 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-ffcdff0e-4d03-4842-a690-c4b497632c24 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] 10.222.0.26 "GET /compute/v2.1/servers/d2b07e55-0b24-4055-889f-e69cdf22f251" status: 200 len: 1267 microversion: 2.1 time: 0.240692 Oct 19 00:47:35.511681 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 661/776] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:47:35 2022] GET /compute/v2.1/servers/d2b07e55-0b24-4055-889f-e69cdf22f251 => generated 1267 bytes in 243 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 19 00:47:35.896491 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-6c3862df-581a-40aa-a6b9-9655d2bbff44 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:47:35.897026 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-6c3862df-581a-40aa-a6b9-9655d2bbff44 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] [instance: b38d4219-8912-4fd9-9d02-7848016f7954] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:47:35.903477 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-6c3862df-581a-40aa-a6b9-9655d2bbff44 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:47:35.903963 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-6c3862df-581a-40aa-a6b9-9655d2bbff44 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:47:35.904398 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-6c3862df-581a-40aa-a6b9-9655d2bbff44 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:47:35.905106 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-6c3862df-581a-40aa-a6b9-9655d2bbff44 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:47:35.905558 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-6c3862df-581a-40aa-a6b9-9655d2bbff44 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:47:35.906002 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-6c3862df-581a-40aa-a6b9-9655d2bbff44 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:47:36.075942 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-6c3862df-581a-40aa-a6b9-9655d2bbff44 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a48ed2d9ded741ca91afc1fbbd9c608f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '82d7723ef95b4c3faeac9835eeb233a5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:47:36.085615 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-6c3862df-581a-40aa-a6b9-9655d2bbff44 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] 10.222.0.26 "GET /compute/v2.1/servers/b38d4219-8912-4fd9-9d02-7848016f7954" status: 200 len: 1725 microversion: 2.1 time: 0.193849 Oct 19 00:47:36.086277 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 662/777] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:47:35 2022] GET /compute/v2.1/servers/b38d4219-8912-4fd9-9d02-7848016f7954 => generated 1725 bytes in 195 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 19 00:47:36.101284 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-ae8a9d13-aa3f-4403-9ebb-6d21565bca15 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:47:36.102444 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-ae8a9d13-aa3f-4403-9ebb-6d21565bca15 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] [instance: 0da74d88-6672-4df0-b665-b9eec2d073c5] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:47:36.113540 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ae8a9d13-aa3f-4403-9ebb-6d21565bca15 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:47:36.114192 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ae8a9d13-aa3f-4403-9ebb-6d21565bca15 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:47:36.114795 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ae8a9d13-aa3f-4403-9ebb-6d21565bca15 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:47:36.115801 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ae8a9d13-aa3f-4403-9ebb-6d21565bca15 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:47:36.116370 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ae8a9d13-aa3f-4403-9ebb-6d21565bca15 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:47:36.116895 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ae8a9d13-aa3f-4403-9ebb-6d21565bca15 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:47:36.266991 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-ae8a9d13-aa3f-4403-9ebb-6d21565bca15 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f8725c4d05b4dd79b05b43fba15980b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0b35c2038f6b4d0a9012b004f1416c00', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:47:36.276779 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-ae8a9d13-aa3f-4403-9ebb-6d21565bca15 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] 10.222.0.26 "GET /compute/v2.1/servers/0da74d88-6672-4df0-b665-b9eec2d073c5" status: 200 len: 1680 microversion: 2.1 time: 0.183912 Oct 19 00:47:36.277379 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 663/778] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:47:36 2022] GET /compute/v2.1/servers/0da74d88-6672-4df0-b665-b9eec2d073c5 => generated 1680 bytes in 186 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 19 00:47:36.538141 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-cdee1428-583b-45fc-a2ea-14c5f668e422 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:47:36.538716 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-cdee1428-583b-45fc-a2ea-14c5f668e422 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] [instance: d2b07e55-0b24-4055-889f-e69cdf22f251] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:47:36.545411 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-cdee1428-583b-45fc-a2ea-14c5f668e422 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:47:36.545924 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-cdee1428-583b-45fc-a2ea-14c5f668e422 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:47:36.546345 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-cdee1428-583b-45fc-a2ea-14c5f668e422 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:47:36.547100 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-cdee1428-583b-45fc-a2ea-14c5f668e422 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:47:36.547554 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-cdee1428-583b-45fc-a2ea-14c5f668e422 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:47:36.548007 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-cdee1428-583b-45fc-a2ea-14c5f668e422 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:47:36.735591 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-cdee1428-583b-45fc-a2ea-14c5f668e422 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f738e67d8c2340028da1254b60d7810e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8684504b0eff415b9063cebae21ca667', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:47:36.752832 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-cdee1428-583b-45fc-a2ea-14c5f668e422 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] 10.222.0.26 "GET /compute/v2.1/servers/d2b07e55-0b24-4055-889f-e69cdf22f251" status: 200 len: 1267 microversion: 2.1 time: 0.220754 Oct 19 00:47:36.753481 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 664/779] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:47:36 2022] GET /compute/v2.1/servers/d2b07e55-0b24-4055-889f-e69cdf22f251 => generated 1267 bytes in 223 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:47:37.127460 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-0ac8b318-e1e7-4cbf-9604-041c04076b34 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:47:37.128549 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-0ac8b318-e1e7-4cbf-9604-041c04076b34 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] [instance: b38d4219-8912-4fd9-9d02-7848016f7954] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:47:37.141320 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-0ac8b318-e1e7-4cbf-9604-041c04076b34 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:47:37.142192 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-0ac8b318-e1e7-4cbf-9604-041c04076b34 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:47:37.142820 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-0ac8b318-e1e7-4cbf-9604-041c04076b34 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:47:37.147173 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-0ac8b318-e1e7-4cbf-9604-041c04076b34 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:47:37.147173 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-0ac8b318-e1e7-4cbf-9604-041c04076b34 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:47:37.147173 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-0ac8b318-e1e7-4cbf-9604-041c04076b34 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:47:37.394538 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-0ac8b318-e1e7-4cbf-9604-041c04076b34 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a48ed2d9ded741ca91afc1fbbd9c608f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '82d7723ef95b4c3faeac9835eeb233a5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:47:37.407285 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-0ac8b318-e1e7-4cbf-9604-041c04076b34 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] 10.222.0.26 "GET /compute/v2.1/servers/b38d4219-8912-4fd9-9d02-7848016f7954" status: 200 len: 1725 microversion: 2.1 time: 0.287617 Oct 19 00:47:37.422517 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 665/780] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:47:37 2022] GET /compute/v2.1/servers/b38d4219-8912-4fd9-9d02-7848016f7954 => generated 1725 bytes in 292 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:47:37.422517 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-23c1e701-9d67-478d-8d58-4a4592a18226 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:47:37.422517 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-23c1e701-9d67-478d-8d58-4a4592a18226 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] [instance: 0da74d88-6672-4df0-b665-b9eec2d073c5] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:47:37.433269 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-23c1e701-9d67-478d-8d58-4a4592a18226 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:47:37.434789 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-23c1e701-9d67-478d-8d58-4a4592a18226 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:47:37.435570 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-23c1e701-9d67-478d-8d58-4a4592a18226 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:47:37.437077 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-23c1e701-9d67-478d-8d58-4a4592a18226 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:47:37.437944 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-23c1e701-9d67-478d-8d58-4a4592a18226 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:47:37.438924 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-23c1e701-9d67-478d-8d58-4a4592a18226 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:47:37.654045 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-23c1e701-9d67-478d-8d58-4a4592a18226 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f8725c4d05b4dd79b05b43fba15980b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0b35c2038f6b4d0a9012b004f1416c00', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:47:37.656953 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG dbcounter [-] [125641] Writing DB stats nova_cell1:SELECT=96,nova_cell1:UPDATE=2,nova_cell1:INSERT=2 {{(pid=125641) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 00:47:37.669552 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-23c1e701-9d67-478d-8d58-4a4592a18226 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] 10.222.0.26 "GET /compute/v2.1/servers/0da74d88-6672-4df0-b665-b9eec2d073c5" status: 200 len: 1680 microversion: 2.1 time: 0.258111 Oct 19 00:47:37.670354 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 666/781] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:47:37 2022] GET /compute/v2.1/servers/0da74d88-6672-4df0-b665-b9eec2d073c5 => generated 1680 bytes in 260 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:47:37.682107 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [req-99c4ec22-fc7e-4eb2-9178-bad1510e0557 req-1b38d324-62ce-4fcc-8795-d20936060549 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "d2b07e55-0b24-4055-889f-e69cdf22f251", "name": "network-vif-plugged", "status": "completed", "tag": "94f720c6-9b61-4983-ab49-77c9cdd0fe18"}]} {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 19 00:47:37.704454 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-99c4ec22-fc7e-4eb2-9178-bad1510e0557 req-1b38d324-62ce-4fcc-8795-d20936060549 service nova] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:47:37.705086 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-99c4ec22-fc7e-4eb2-9178-bad1510e0557 req-1b38d324-62ce-4fcc-8795-d20936060549 service nova] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:47:37.705650 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-99c4ec22-fc7e-4eb2-9178-bad1510e0557 req-1b38d324-62ce-4fcc-8795-d20936060549 service nova] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:47:37.721788 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.compute.server_external_events [req-99c4ec22-fc7e-4eb2-9178-bad1510e0557 req-1b38d324-62ce-4fcc-8795-d20936060549 service nova] Creating event network-vif-plugged:94f720c6-9b61-4983-ab49-77c9cdd0fe18 for instance d2b07e55-0b24-4055-889f-e69cdf22f251 on hv-cinder-maste Oct 19 00:47:37.729701 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [req-99c4ec22-fc7e-4eb2-9178-bad1510e0557 req-1b38d324-62ce-4fcc-8795-d20936060549 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.056782 Oct 19 00:47:37.730420 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 667/782] 10.222.0.26 () {64 vars in 1317 bytes} [Wed Oct 19 00:47:37 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 59 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 19 00:47:37.806668 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-c808e448-8a2c-4768-be51-1394a042cf8f tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:47:37.808000 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-c808e448-8a2c-4768-be51-1394a042cf8f tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] [instance: d2b07e55-0b24-4055-889f-e69cdf22f251] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:47:37.821995 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-c808e448-8a2c-4768-be51-1394a042cf8f tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:47:37.823030 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-c808e448-8a2c-4768-be51-1394a042cf8f tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:47:37.823906 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-c808e448-8a2c-4768-be51-1394a042cf8f tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:47:37.825231 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-c808e448-8a2c-4768-be51-1394a042cf8f tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:47:37.826040 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-c808e448-8a2c-4768-be51-1394a042cf8f tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:47:37.826821 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-c808e448-8a2c-4768-be51-1394a042cf8f tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:47:37.997983 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-c808e448-8a2c-4768-be51-1394a042cf8f tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f738e67d8c2340028da1254b60d7810e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8684504b0eff415b9063cebae21ca667', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:47:38.013956 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-c808e448-8a2c-4768-be51-1394a042cf8f tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] 10.222.0.26 "GET /compute/v2.1/servers/d2b07e55-0b24-4055-889f-e69cdf22f251" status: 200 len: 1267 microversion: 2.1 time: 0.218936 Oct 19 00:47:38.015080 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 668/783] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:47:37 2022] GET /compute/v2.1/servers/d2b07e55-0b24-4055-889f-e69cdf22f251 => generated 1267 bytes in 222 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 19 00:47:38.439358 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-74889410-c455-4aac-8aec-8e9b22e68ca6 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:47:38.440393 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-74889410-c455-4aac-8aec-8e9b22e68ca6 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] [instance: b38d4219-8912-4fd9-9d02-7848016f7954] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:47:38.451453 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-74889410-c455-4aac-8aec-8e9b22e68ca6 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:47:38.455219 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-74889410-c455-4aac-8aec-8e9b22e68ca6 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:47:38.455219 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-74889410-c455-4aac-8aec-8e9b22e68ca6 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:47:38.455219 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-74889410-c455-4aac-8aec-8e9b22e68ca6 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:47:38.455219 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-74889410-c455-4aac-8aec-8e9b22e68ca6 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:47:38.455949 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-74889410-c455-4aac-8aec-8e9b22e68ca6 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:47:38.713502 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-74889410-c455-4aac-8aec-8e9b22e68ca6 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a48ed2d9ded741ca91afc1fbbd9c608f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '82d7723ef95b4c3faeac9835eeb233a5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:47:38.723450 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-74889410-c455-4aac-8aec-8e9b22e68ca6 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] 10.222.0.26 "GET /compute/v2.1/servers/b38d4219-8912-4fd9-9d02-7848016f7954" status: 200 len: 1725 microversion: 2.1 time: 0.290498 Oct 19 00:47:38.733633 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 669/784] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:47:38 2022] GET /compute/v2.1/servers/b38d4219-8912-4fd9-9d02-7848016f7954 => generated 1725 bytes in 300 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 19 00:47:38.744523 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [req-f692e515-1ba8-4beb-8f30-10f0b44f7e18 req-3df4ff41-5539-416b-b867-d5df177b91df service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "0da74d88-6672-4df0-b665-b9eec2d073c5", "name": "network-vif-deleted", "tag": "6ccabad6-2670-419e-95ab-c0c5d0749b3a"}]} {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 19 00:47:38.764980 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-f692e515-1ba8-4beb-8f30-10f0b44f7e18 req-3df4ff41-5539-416b-b867-d5df177b91df service nova] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:47:38.765938 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-f692e515-1ba8-4beb-8f30-10f0b44f7e18 req-3df4ff41-5539-416b-b867-d5df177b91df service nova] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:47:38.766785 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-f692e515-1ba8-4beb-8f30-10f0b44f7e18 req-3df4ff41-5539-416b-b867-d5df177b91df service nova] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:47:38.789923 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.compute.server_external_events [req-f692e515-1ba8-4beb-8f30-10f0b44f7e18 req-3df4ff41-5539-416b-b867-d5df177b91df service nova] Creating event network-vif-deleted:6ccabad6-2670-419e-95ab-c0c5d0749b3a for instance 0da74d88-6672-4df0-b665-b9eec2d073c5 on hv-cinder-maste Oct 19 00:47:38.798345 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [req-f692e515-1ba8-4beb-8f30-10f0b44f7e18 req-3df4ff41-5539-416b-b867-d5df177b91df service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.062076 Oct 19 00:47:38.799514 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 670/785] 10.222.0.26 () {64 vars in 1317 bytes} [Wed Oct 19 00:47:38 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 65 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 19 00:47:38.810434 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-1061c4bc-0acb-4396-b09a-77d2824e61ea tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:47:38.811795 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-1061c4bc-0acb-4396-b09a-77d2824e61ea tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] [instance: 0da74d88-6672-4df0-b665-b9eec2d073c5] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:47:38.823859 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-1061c4bc-0acb-4396-b09a-77d2824e61ea tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:47:38.824770 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-1061c4bc-0acb-4396-b09a-77d2824e61ea tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:47:38.825558 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-1061c4bc-0acb-4396-b09a-77d2824e61ea tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:47:38.826889 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-1061c4bc-0acb-4396-b09a-77d2824e61ea tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:47:38.827723 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-1061c4bc-0acb-4396-b09a-77d2824e61ea tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:47:38.828489 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-1061c4bc-0acb-4396-b09a-77d2824e61ea tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:47:38.940444 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-1061c4bc-0acb-4396-b09a-77d2824e61ea tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f8725c4d05b4dd79b05b43fba15980b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0b35c2038f6b4d0a9012b004f1416c00', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:47:38.961905 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-1061c4bc-0acb-4396-b09a-77d2824e61ea tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] 10.222.0.26 "GET /compute/v2.1/servers/0da74d88-6672-4df0-b665-b9eec2d073c5" status: 200 len: 1612 microversion: 2.1 time: 0.158863 Oct 19 00:47:38.982822 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 671/786] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:47:38 2022] GET /compute/v2.1/servers/0da74d88-6672-4df0-b665-b9eec2d073c5 => generated 1612 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:47:39.041133 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-8fdd1142-73c5-4b04-b62c-1aa042c1418b tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:47:39.041948 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-8fdd1142-73c5-4b04-b62c-1aa042c1418b tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] [instance: d2b07e55-0b24-4055-889f-e69cdf22f251] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:47:39.054748 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-8fdd1142-73c5-4b04-b62c-1aa042c1418b tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:47:39.055752 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-8fdd1142-73c5-4b04-b62c-1aa042c1418b tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:47:39.056626 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-8fdd1142-73c5-4b04-b62c-1aa042c1418b tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:47:39.058039 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-8fdd1142-73c5-4b04-b62c-1aa042c1418b tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:47:39.059348 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-8fdd1142-73c5-4b04-b62c-1aa042c1418b tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:47:39.059899 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-8fdd1142-73c5-4b04-b62c-1aa042c1418b tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:47:39.258302 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-8fdd1142-73c5-4b04-b62c-1aa042c1418b tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f738e67d8c2340028da1254b60d7810e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8684504b0eff415b9063cebae21ca667', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:47:39.273385 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-8fdd1142-73c5-4b04-b62c-1aa042c1418b tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] 10.222.0.26 "GET /compute/v2.1/servers/d2b07e55-0b24-4055-889f-e69cdf22f251" status: 200 len: 1267 microversion: 2.1 time: 0.236230 Oct 19 00:47:39.274439 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 672/787] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:47:39 2022] GET /compute/v2.1/servers/d2b07e55-0b24-4055-889f-e69cdf22f251 => generated 1267 bytes in 239 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 19 00:47:39.749782 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-ecb45f6d-fdb7-42b3-adec-b07e4749a89f tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:47:39.751013 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-ecb45f6d-fdb7-42b3-adec-b07e4749a89f tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] [instance: b38d4219-8912-4fd9-9d02-7848016f7954] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:47:39.760273 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ecb45f6d-fdb7-42b3-adec-b07e4749a89f tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:47:39.760908 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ecb45f6d-fdb7-42b3-adec-b07e4749a89f tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:47:39.761595 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ecb45f6d-fdb7-42b3-adec-b07e4749a89f tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:47:39.762799 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ecb45f6d-fdb7-42b3-adec-b07e4749a89f tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:47:39.763518 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ecb45f6d-fdb7-42b3-adec-b07e4749a89f tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:47:39.764825 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ecb45f6d-fdb7-42b3-adec-b07e4749a89f tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:47:40.007951 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-ecb45f6d-fdb7-42b3-adec-b07e4749a89f tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a48ed2d9ded741ca91afc1fbbd9c608f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '82d7723ef95b4c3faeac9835eeb233a5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:47:40.026947 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-ecb45f6d-fdb7-42b3-adec-b07e4749a89f tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] 10.222.0.26 "GET /compute/v2.1/servers/b38d4219-8912-4fd9-9d02-7848016f7954" status: 200 len: 1725 microversion: 2.1 time: 0.283231 Oct 19 00:47:40.028145 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 673/788] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:47:39 2022] GET /compute/v2.1/servers/b38d4219-8912-4fd9-9d02-7848016f7954 => generated 1725 bytes in 286 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 19 00:47:40.051904 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-ea9cfdfd-0890-47d0-8e53-505e0fd74496 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:47:40.055958 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-ea9cfdfd-0890-47d0-8e53-505e0fd74496 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] [instance: 0da74d88-6672-4df0-b665-b9eec2d073c5] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:47:40.068897 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ea9cfdfd-0890-47d0-8e53-505e0fd74496 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:47:40.069767 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ea9cfdfd-0890-47d0-8e53-505e0fd74496 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:47:40.070455 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ea9cfdfd-0890-47d0-8e53-505e0fd74496 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:47:40.072093 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ea9cfdfd-0890-47d0-8e53-505e0fd74496 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:47:40.072888 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ea9cfdfd-0890-47d0-8e53-505e0fd74496 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:47:40.073608 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ea9cfdfd-0890-47d0-8e53-505e0fd74496 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:47:40.113644 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ea9cfdfd-0890-47d0-8e53-505e0fd74496 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:47:40.114141 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ea9cfdfd-0890-47d0-8e53-505e0fd74496 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:47:40.114589 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ea9cfdfd-0890-47d0-8e53-505e0fd74496 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:47:40.115056 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.objects.instance [None req-ea9cfdfd-0890-47d0-8e53-505e0fd74496 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lazy-loading 'fault' on Instance uuid 0da74d88-6672-4df0-b665-b9eec2d073c5 {{(pid=125641) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 19 00:47:40.173971 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-ea9cfdfd-0890-47d0-8e53-505e0fd74496 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6f8725c4d05b4dd79b05b43fba15980b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0b35c2038f6b4d0a9012b004f1416c00', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:47:40.185345 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-ea9cfdfd-0890-47d0-8e53-505e0fd74496 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] 10.222.0.26 "GET /compute/v2.1/servers/0da74d88-6672-4df0-b665-b9eec2d073c5" status: 200 len: 1331 microversion: 2.1 time: 0.140373 Oct 19 00:47:40.185950 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 674/789] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:47:40 2022] GET /compute/v2.1/servers/0da74d88-6672-4df0-b665-b9eec2d073c5 => generated 1331 bytes in 142 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:47:40.309641 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-5c61905f-aa0d-4aa5-b324-2416fcec4742 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:47:40.310258 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-5c61905f-aa0d-4aa5-b324-2416fcec4742 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] [instance: d2b07e55-0b24-4055-889f-e69cdf22f251] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:47:40.317043 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-5c61905f-aa0d-4aa5-b324-2416fcec4742 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:47:40.317538 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-5c61905f-aa0d-4aa5-b324-2416fcec4742 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:47:40.317987 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-5c61905f-aa0d-4aa5-b324-2416fcec4742 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:47:40.318730 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-5c61905f-aa0d-4aa5-b324-2416fcec4742 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:47:40.319260 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-5c61905f-aa0d-4aa5-b324-2416fcec4742 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:47:40.319601 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-5c61905f-aa0d-4aa5-b324-2416fcec4742 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:47:40.499494 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-5c61905f-aa0d-4aa5-b324-2416fcec4742 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f738e67d8c2340028da1254b60d7810e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8684504b0eff415b9063cebae21ca667', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:47:40.508692 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-5c61905f-aa0d-4aa5-b324-2416fcec4742 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] 10.222.0.26 "GET /compute/v2.1/servers/d2b07e55-0b24-4055-889f-e69cdf22f251" status: 200 len: 1455 microversion: 2.1 time: 0.203028 Oct 19 00:47:40.509356 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 675/790] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:47:40 2022] GET /compute/v2.1/servers/d2b07e55-0b24-4055-889f-e69cdf22f251 => generated 1455 bytes in 204 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 19 00:47:40.532371 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-64effed7-3974-4d81-8526-de9811fd5b49 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:47:40.532985 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-64effed7-3974-4d81-8526-de9811fd5b49 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] [instance: d2b07e55-0b24-4055-889f-e69cdf22f251] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:47:40.539976 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-64effed7-3974-4d81-8526-de9811fd5b49 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:47:40.540515 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-64effed7-3974-4d81-8526-de9811fd5b49 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:47:40.541059 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-64effed7-3974-4d81-8526-de9811fd5b49 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:47:40.542106 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-64effed7-3974-4d81-8526-de9811fd5b49 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:47:40.542831 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-64effed7-3974-4d81-8526-de9811fd5b49 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:47:40.543502 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-64effed7-3974-4d81-8526-de9811fd5b49 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:47:40.721230 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-64effed7-3974-4d81-8526-de9811fd5b49 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f738e67d8c2340028da1254b60d7810e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8684504b0eff415b9063cebae21ca667', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:47:40.731516 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-64effed7-3974-4d81-8526-de9811fd5b49 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] 10.222.0.26 "GET /compute/v2.1/servers/d2b07e55-0b24-4055-889f-e69cdf22f251" status: 200 len: 1455 microversion: 2.1 time: 0.202925 Oct 19 00:47:40.732296 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 676/791] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:47:40 2022] GET /compute/v2.1/servers/d2b07e55-0b24-4055-889f-e69cdf22f251 => generated 1455 bytes in 205 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 19 00:47:41.064462 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-cd33b8b1-6c28-442a-81ad-c56e286c9d0d tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:47:41.065686 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-cd33b8b1-6c28-442a-81ad-c56e286c9d0d tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] [instance: b38d4219-8912-4fd9-9d02-7848016f7954] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:47:41.080305 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-cd33b8b1-6c28-442a-81ad-c56e286c9d0d tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:47:41.081401 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-cd33b8b1-6c28-442a-81ad-c56e286c9d0d tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:47:41.085360 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-cd33b8b1-6c28-442a-81ad-c56e286c9d0d tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:47:41.086212 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-cd33b8b1-6c28-442a-81ad-c56e286c9d0d tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:47:41.087390 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-cd33b8b1-6c28-442a-81ad-c56e286c9d0d tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:47:41.088153 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-cd33b8b1-6c28-442a-81ad-c56e286c9d0d tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:47:41.476442 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-cd33b8b1-6c28-442a-81ad-c56e286c9d0d tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a48ed2d9ded741ca91afc1fbbd9c608f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '82d7723ef95b4c3faeac9835eeb233a5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:47:41.487399 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-cd33b8b1-6c28-442a-81ad-c56e286c9d0d tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] 10.222.0.26 "GET /compute/v2.1/servers/b38d4219-8912-4fd9-9d02-7848016f7954" status: 200 len: 1725 microversion: 2.1 time: 0.430407 Oct 19 00:47:41.488962 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 677/792] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:47:41 2022] GET /compute/v2.1/servers/b38d4219-8912-4fd9-9d02-7848016f7954 => generated 1725 bytes in 433 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 19 00:47:41.493514 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-a6dbeb6f-007e-4814-905e-8e0cb20a297b tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:47:41.494627 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-a6dbeb6f-007e-4814-905e-8e0cb20a297b tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] [instance: 0da74d88-6672-4df0-b665-b9eec2d073c5] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:47:41.500863 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-a6dbeb6f-007e-4814-905e-8e0cb20a297b tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:47:41.501418 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-a6dbeb6f-007e-4814-905e-8e0cb20a297b tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:47:41.501800 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-a6dbeb6f-007e-4814-905e-8e0cb20a297b tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:47:41.502566 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-a6dbeb6f-007e-4814-905e-8e0cb20a297b tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:47:41.503060 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-a6dbeb6f-007e-4814-905e-8e0cb20a297b tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:47:41.504217 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-a6dbeb6f-007e-4814-905e-8e0cb20a297b tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:47:41.520872 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.wsgi [None req-a6dbeb6f-007e-4814-905e-8e0cb20a297b tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] HTTP exception thrown: Instance 0da74d88-6672-4df0-b665-b9eec2d073c5 could not be found. Oct 19 00:47:41.522655 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-a6dbeb6f-007e-4814-905e-8e0cb20a297b tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Request method failure captured: Oct 19 00:47:41.522655 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: request: GET /compute/v2.1/servers/0da74d88-6672-4df0-b665-b9eec2d073c5 HTTP/1.1 Oct 19 00:47:41.522655 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Accept: application/json Oct 19 00:47:41.522655 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Accept-Encoding: identity Oct 19 00:47:41.522655 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Connection: close Oct 19 00:47:41.522655 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Content-Length: 0 Oct 19 00:47:41.522655 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Content-Type: application/json Oct 19 00:47:41.522655 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Host: 10.222.0.26 Oct 19 00:47:41.522655 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Openstack-System-Scope: None Oct 19 00:47:41.522655 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: User-Agent: python-urllib3/1.26.12 Oct 19 00:47:41.522655 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Auth-Token: gAAAAABjT0eJXd9ar1txWHw929SSSa4BnzN-f-jxcCDsiqb5Qu6eweguqGUJXghFM8xaTx1ciC3SJFoac87bZHlU5VANYjjzV6ICQChedig2cB1RmNul9h3AYlunCOiAit1Pl73UlpO5HCAxVXGmz0N3G59VsZacZSMYF6RytJzlde2rg7lCpks Oct 19 00:47:41.522655 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Domain-Id: None Oct 19 00:47:41.522655 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Domain-Name: None Oct 19 00:47:41.522655 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Identity-Status: Confirmed Oct 19 00:47:41.522655 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Is-Admin-Project: True Oct 19 00:47:41.522655 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Project-Domain-Id: default Oct 19 00:47:41.522655 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Project-Domain-Name: Default Oct 19 00:47:41.522655 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Project-Id: 0b35c2038f6b4d0a9012b004f1416c00 Oct 19 00:47:41.522655 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Project-Name: tempest-AttachVolumeNegativeTest-1282551484 Oct 19 00:47:41.522655 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Role: member,reader Oct 19 00:47:41.522655 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Roles: member,reader Oct 19 00:47:41.525306 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/0b35c2038f6b4d0a9012b004f1416c00"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/0b35c2038f6b4d0a9012b004f1416c00"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/0b35c2038f6b4d0a9012b004f1416c00"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_0b35c2038f6b4d0a9012b004f1416c00"}]}] Oct 19 00:47:41.525306 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Tenant: tempest-AttachVolumeNegativeTest-1282551484 Oct 19 00:47:41.525306 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Tenant-Id: 0b35c2038f6b4d0a9012b004f1416c00 Oct 19 00:47:41.525306 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1282551484 Oct 19 00:47:41.525306 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User: tempest-AttachVolumeNegativeTest-1282551484-project Oct 19 00:47:41.525306 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User-Domain-Id: default Oct 19 00:47:41.525306 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User-Domain-Name: Default Oct 19 00:47:41.525306 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User-Id: 6f8725c4d05b4dd79b05b43fba15980b Oct 19 00:47:41.525306 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User-Name: tempest-AttachVolumeNegativeTest-1282551484-project Oct 19 00:47:41.525306 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: method: > Oct 19 00:47:41.525306 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: exception: Instance 0da74d88-6672-4df0-b665-b9eec2d073c5 could not be found. Oct 19 00:47:41.525306 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: action_args: {'id': '0da74d88-6672-4df0-b665-b9eec2d073c5'} Oct 19 00:47:41.527588 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 19 00:47:41.527588 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 19 00:47:41.527588 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 19 00:47:41.527588 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 19 00:47:41.527588 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 19 00:47:41.527588 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 19 00:47:41.527588 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 19 00:47:41.527588 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 19 00:47:41.527588 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 19 00:47:41.527588 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 19 00:47:41.527588 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 0da74d88-6672-4df0-b665-b9eec2d073c5 could not be found. Oct 19 00:47:41.527588 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:47:41.527588 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 19 00:47:41.527588 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:47:41.527588 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 19 00:47:41.527588 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 19 00:47:41.527588 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 19 00:47:41.527588 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 19 00:47:41.527588 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 19 00:47:41.527588 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 19 00:47:41.530270 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 19 00:47:41.530270 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 19 00:47:41.530270 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 19 00:47:41.530270 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 19 00:47:41.530270 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 19 00:47:41.530270 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 19 00:47:41.530270 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 19 00:47:41.530270 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 0da74d88-6672-4df0-b665-b9eec2d073c5 could not be found. Oct 19 00:47:41.530270 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:47:41.530270 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 19 00:47:41.530270 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:47:41.530270 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 19 00:47:41.530270 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 19 00:47:41.530270 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 19 00:47:41.530270 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 19 00:47:41.530270 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 19 00:47:41.530270 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 0da74d88-6672-4df0-b665-b9eec2d073c5 could not be found. Oct 19 00:47:41.530270 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:47:41.532981 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-a6dbeb6f-007e-4814-905e-8e0cb20a297b tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Returning 404 to user: Instance 0da74d88-6672-4df0-b665-b9eec2d073c5 could not be found. {{(pid=125641) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 19 00:47:41.532981 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-a6dbeb6f-007e-4814-905e-8e0cb20a297b tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] 10.222.0.26 "GET /compute/v2.1/servers/0da74d88-6672-4df0-b665-b9eec2d073c5" status: 404 len: 111 microversion: 2.1 time: 0.034505 Oct 19 00:47:41.532981 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 678/793] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:47:41 2022] GET /compute/v2.1/servers/0da74d88-6672-4df0-b665-b9eec2d073c5 => generated 111 bytes in 35 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 19 00:47:41.537172 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-5250535f-4db9-4646-9c83-a687a1ee5816 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Calling method '.version_select at 0x7f71c3f0e700>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:47:41.572425 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-5250535f-4db9-4646-9c83-a687a1ee5816 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] 10.222.0.26 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-1023768314" status: 202 len: 0 microversion: 2.1 time: 0.039845 Oct 19 00:47:41.573515 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 679/794] 10.222.0.26 () {58 vars in 1221 bytes} [Wed Oct 19 00:47:41 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-1023768314 => generated 0 bytes in 44 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Oct 19 00:47:42.520130 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-59a02118-35e4-4ec2-b392-50840e68d7a6 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:47:42.521746 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-59a02118-35e4-4ec2-b392-50840e68d7a6 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] [instance: b38d4219-8912-4fd9-9d02-7848016f7954] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:47:42.529710 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-59a02118-35e4-4ec2-b392-50840e68d7a6 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:47:42.531186 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-59a02118-35e4-4ec2-b392-50840e68d7a6 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:47:42.531937 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-59a02118-35e4-4ec2-b392-50840e68d7a6 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:47:42.532959 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-59a02118-35e4-4ec2-b392-50840e68d7a6 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:47:42.533626 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-59a02118-35e4-4ec2-b392-50840e68d7a6 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:47:42.534247 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-59a02118-35e4-4ec2-b392-50840e68d7a6 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:47:43.101152 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-59a02118-35e4-4ec2-b392-50840e68d7a6 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a48ed2d9ded741ca91afc1fbbd9c608f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '82d7723ef95b4c3faeac9835eeb233a5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:47:43.117202 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-59a02118-35e4-4ec2-b392-50840e68d7a6 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] 10.222.0.26 "GET /compute/v2.1/servers/b38d4219-8912-4fd9-9d02-7848016f7954" status: 200 len: 1725 microversion: 2.1 time: 0.602007 Oct 19 00:47:43.118208 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 680/795] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:47:42 2022] GET /compute/v2.1/servers/b38d4219-8912-4fd9-9d02-7848016f7954 => generated 1725 bytes in 604 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 19 00:47:43.152577 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Oct 19 00:47:43.153199 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: Wed Oct 19 00:47:43 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers (ip 10.222.0.26) !!! Oct 19 00:47:43.434003 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [req-8bcb874b-00d4-4fd9-936a-001962f3c88d req-9ebd6b30-abe6-4443-8912-548e6a2d8d7f service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "d2b07e55-0b24-4055-889f-e69cdf22f251", "tag": "94f720c6-9b61-4983-ab49-77c9cdd0fe18"}]} {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 19 00:47:43.466092 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-8bcb874b-00d4-4fd9-936a-001962f3c88d req-9ebd6b30-abe6-4443-8912-548e6a2d8d7f service nova] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:47:43.466842 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-8bcb874b-00d4-4fd9-936a-001962f3c88d req-9ebd6b30-abe6-4443-8912-548e6a2d8d7f service nova] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:47:43.467632 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-8bcb874b-00d4-4fd9-936a-001962f3c88d req-9ebd6b30-abe6-4443-8912-548e6a2d8d7f service nova] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:47:43.486607 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.compute.server_external_events [req-8bcb874b-00d4-4fd9-936a-001962f3c88d req-9ebd6b30-abe6-4443-8912-548e6a2d8d7f service nova] Creating event network-changed:94f720c6-9b61-4983-ab49-77c9cdd0fe18 for instance d2b07e55-0b24-4055-889f-e69cdf22f251 on hv-cinder-maste Oct 19 00:47:43.492349 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [req-8bcb874b-00d4-4fd9-936a-001962f3c88d req-9ebd6b30-abe6-4443-8912-548e6a2d8d7f service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.065389 Oct 19 00:47:43.493339 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 681/796] 10.222.0.26 () {64 vars in 1317 bytes} [Wed Oct 19 00:47:43 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 68 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 19 00:47:43.766355 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-7c923452-8758-45cd-aa31-8a18d9c0aeb4 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:47:43.767515 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-7c923452-8758-45cd-aa31-8a18d9c0aeb4 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] [instance: 0da74d88-6672-4df0-b665-b9eec2d073c5] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:47:43.780479 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-7c923452-8758-45cd-aa31-8a18d9c0aeb4 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:47:43.781337 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-7c923452-8758-45cd-aa31-8a18d9c0aeb4 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:47:43.782121 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-7c923452-8758-45cd-aa31-8a18d9c0aeb4 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:47:43.784293 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-7c923452-8758-45cd-aa31-8a18d9c0aeb4 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:47:43.784293 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-7c923452-8758-45cd-aa31-8a18d9c0aeb4 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:47:43.784875 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-7c923452-8758-45cd-aa31-8a18d9c0aeb4 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:47:43.813937 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.wsgi [None req-7c923452-8758-45cd-aa31-8a18d9c0aeb4 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] HTTP exception thrown: Instance 0da74d88-6672-4df0-b665-b9eec2d073c5 could not be found. Oct 19 00:47:43.817710 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-7c923452-8758-45cd-aa31-8a18d9c0aeb4 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Request method failure captured: Oct 19 00:47:43.817710 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: request: DELETE /compute/v2.1/servers/0da74d88-6672-4df0-b665-b9eec2d073c5 HTTP/1.1 Oct 19 00:47:43.817710 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Accept: application/json Oct 19 00:47:43.817710 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Accept-Encoding: identity Oct 19 00:47:43.817710 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Connection: close Oct 19 00:47:43.817710 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Content-Length: 0 Oct 19 00:47:43.817710 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Content-Type: application/json Oct 19 00:47:43.817710 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Host: 10.222.0.26 Oct 19 00:47:43.817710 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Openstack-System-Scope: None Oct 19 00:47:43.817710 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: User-Agent: python-urllib3/1.26.12 Oct 19 00:47:43.817710 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Auth-Token: gAAAAABjT0eJXd9ar1txWHw929SSSa4BnzN-f-jxcCDsiqb5Qu6eweguqGUJXghFM8xaTx1ciC3SJFoac87bZHlU5VANYjjzV6ICQChedig2cB1RmNul9h3AYlunCOiAit1Pl73UlpO5HCAxVXGmz0N3G59VsZacZSMYF6RytJzlde2rg7lCpks Oct 19 00:47:43.817710 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Domain-Id: None Oct 19 00:47:43.817710 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Domain-Name: None Oct 19 00:47:43.817710 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Identity-Status: Confirmed Oct 19 00:47:43.817710 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Is-Admin-Project: True Oct 19 00:47:43.817710 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Project-Domain-Id: default Oct 19 00:47:43.817710 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Project-Domain-Name: Default Oct 19 00:47:43.817710 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Project-Id: 0b35c2038f6b4d0a9012b004f1416c00 Oct 19 00:47:43.817710 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Project-Name: tempest-AttachVolumeNegativeTest-1282551484 Oct 19 00:47:43.817710 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Role: member,reader Oct 19 00:47:43.817710 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Roles: member,reader Oct 19 00:47:43.820399 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/0b35c2038f6b4d0a9012b004f1416c00"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/0b35c2038f6b4d0a9012b004f1416c00"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/0b35c2038f6b4d0a9012b004f1416c00"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_0b35c2038f6b4d0a9012b004f1416c00"}]}] Oct 19 00:47:43.820399 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Tenant: tempest-AttachVolumeNegativeTest-1282551484 Oct 19 00:47:43.820399 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Tenant-Id: 0b35c2038f6b4d0a9012b004f1416c00 Oct 19 00:47:43.820399 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1282551484 Oct 19 00:47:43.820399 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User: tempest-AttachVolumeNegativeTest-1282551484-project Oct 19 00:47:43.820399 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User-Domain-Id: default Oct 19 00:47:43.820399 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User-Domain-Name: Default Oct 19 00:47:43.820399 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User-Id: 6f8725c4d05b4dd79b05b43fba15980b Oct 19 00:47:43.820399 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User-Name: tempest-AttachVolumeNegativeTest-1282551484-project Oct 19 00:47:43.820399 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: method: > Oct 19 00:47:43.820399 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: exception: Instance 0da74d88-6672-4df0-b665-b9eec2d073c5 could not be found. Oct 19 00:47:43.820399 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: action_args: {'id': '0da74d88-6672-4df0-b665-b9eec2d073c5'} Oct 19 00:47:43.821842 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 19 00:47:43.821842 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 19 00:47:43.821842 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 19 00:47:43.821842 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 19 00:47:43.821842 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 19 00:47:43.821842 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 19 00:47:43.821842 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 19 00:47:43.821842 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 19 00:47:43.821842 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 19 00:47:43.821842 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 19 00:47:43.821842 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 0da74d88-6672-4df0-b665-b9eec2d073c5 could not be found. Oct 19 00:47:43.821842 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:47:43.821842 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 19 00:47:43.821842 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:47:43.821842 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 19 00:47:43.821842 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 19 00:47:43.821842 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 19 00:47:43.821842 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 19 00:47:43.821842 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 19 00:47:43.821842 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 19 00:47:43.828385 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 19 00:47:43.828385 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Oct 19 00:47:43.828385 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Oct 19 00:47:43.828385 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Oct 19 00:47:43.828385 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Oct 19 00:47:43.828385 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 19 00:47:43.828385 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 19 00:47:43.828385 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 19 00:47:43.828385 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 19 00:47:43.828385 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 0da74d88-6672-4df0-b665-b9eec2d073c5 could not be found. Oct 19 00:47:43.828385 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:47:43.828385 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 19 00:47:43.828385 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:47:43.828385 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 19 00:47:43.828385 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 19 00:47:43.828385 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 19 00:47:43.828385 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 19 00:47:43.828385 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 19 00:47:43.828385 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 0da74d88-6672-4df0-b665-b9eec2d073c5 could not be found. Oct 19 00:47:43.828385 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:47:43.829809 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-7c923452-8758-45cd-aa31-8a18d9c0aeb4 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Returning 404 to user: Instance 0da74d88-6672-4df0-b665-b9eec2d073c5 could not be found. {{(pid=125641) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 19 00:47:43.829809 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-7c923452-8758-45cd-aa31-8a18d9c0aeb4 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] 10.222.0.26 "DELETE /compute/v2.1/servers/0da74d88-6672-4df0-b665-b9eec2d073c5" status: 404 len: 111 microversion: 2.1 time: 0.062289 Oct 19 00:47:43.829809 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 682/797] 10.222.0.26 () {58 vars in 1239 bytes} [Wed Oct 19 00:47:43 2022] DELETE /compute/v2.1/servers/0da74d88-6672-4df0-b665-b9eec2d073c5 => generated 111 bytes in 65 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 19 00:47:43.836900 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-471c591a-5b2d-498c-98a9-3cc5df53e2b1 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:47:43.839288 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-471c591a-5b2d-498c-98a9-3cc5df53e2b1 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] [instance: 0da74d88-6672-4df0-b665-b9eec2d073c5] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:47:43.847771 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-471c591a-5b2d-498c-98a9-3cc5df53e2b1 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:47:43.848484 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-471c591a-5b2d-498c-98a9-3cc5df53e2b1 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:47:43.849175 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-471c591a-5b2d-498c-98a9-3cc5df53e2b1 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:47:43.850062 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-471c591a-5b2d-498c-98a9-3cc5df53e2b1 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:47:43.850864 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-471c591a-5b2d-498c-98a9-3cc5df53e2b1 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:47:43.851456 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-471c591a-5b2d-498c-98a9-3cc5df53e2b1 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:47:43.872977 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.wsgi [None req-471c591a-5b2d-498c-98a9-3cc5df53e2b1 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] HTTP exception thrown: Instance 0da74d88-6672-4df0-b665-b9eec2d073c5 could not be found. Oct 19 00:47:43.876018 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-471c591a-5b2d-498c-98a9-3cc5df53e2b1 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Request method failure captured: Oct 19 00:47:43.876018 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: request: GET /compute/v2.1/servers/0da74d88-6672-4df0-b665-b9eec2d073c5 HTTP/1.1 Oct 19 00:47:43.876018 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Accept: application/json Oct 19 00:47:43.876018 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Accept-Encoding: identity Oct 19 00:47:43.876018 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Connection: close Oct 19 00:47:43.876018 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Content-Length: 0 Oct 19 00:47:43.876018 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Content-Type: application/json Oct 19 00:47:43.876018 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Host: 10.222.0.26 Oct 19 00:47:43.876018 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Openstack-System-Scope: None Oct 19 00:47:43.876018 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: User-Agent: python-urllib3/1.26.12 Oct 19 00:47:43.876018 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Auth-Token: gAAAAABjT0eJXd9ar1txWHw929SSSa4BnzN-f-jxcCDsiqb5Qu6eweguqGUJXghFM8xaTx1ciC3SJFoac87bZHlU5VANYjjzV6ICQChedig2cB1RmNul9h3AYlunCOiAit1Pl73UlpO5HCAxVXGmz0N3G59VsZacZSMYF6RytJzlde2rg7lCpks Oct 19 00:47:43.876018 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Domain-Id: None Oct 19 00:47:43.876018 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Domain-Name: None Oct 19 00:47:43.876018 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Identity-Status: Confirmed Oct 19 00:47:43.876018 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Is-Admin-Project: True Oct 19 00:47:43.876018 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Project-Domain-Id: default Oct 19 00:47:43.876018 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Project-Domain-Name: Default Oct 19 00:47:43.876018 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Project-Id: 0b35c2038f6b4d0a9012b004f1416c00 Oct 19 00:47:43.876018 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Project-Name: tempest-AttachVolumeNegativeTest-1282551484 Oct 19 00:47:43.876018 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Role: member,reader Oct 19 00:47:43.876018 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Roles: member,reader Oct 19 00:47:43.877488 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/0b35c2038f6b4d0a9012b004f1416c00"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/0b35c2038f6b4d0a9012b004f1416c00"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/0b35c2038f6b4d0a9012b004f1416c00"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_0b35c2038f6b4d0a9012b004f1416c00"}]}] Oct 19 00:47:43.877488 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Tenant: tempest-AttachVolumeNegativeTest-1282551484 Oct 19 00:47:43.877488 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Tenant-Id: 0b35c2038f6b4d0a9012b004f1416c00 Oct 19 00:47:43.877488 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1282551484 Oct 19 00:47:43.877488 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User: tempest-AttachVolumeNegativeTest-1282551484-project Oct 19 00:47:43.877488 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User-Domain-Id: default Oct 19 00:47:43.877488 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User-Domain-Name: Default Oct 19 00:47:43.877488 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User-Id: 6f8725c4d05b4dd79b05b43fba15980b Oct 19 00:47:43.877488 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User-Name: tempest-AttachVolumeNegativeTest-1282551484-project Oct 19 00:47:43.877488 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: method: > Oct 19 00:47:43.877488 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: exception: Instance 0da74d88-6672-4df0-b665-b9eec2d073c5 could not be found. Oct 19 00:47:43.877488 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: action_args: {'id': '0da74d88-6672-4df0-b665-b9eec2d073c5'} Oct 19 00:47:43.878821 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 19 00:47:43.878821 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 19 00:47:43.878821 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 19 00:47:43.878821 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 19 00:47:43.878821 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 19 00:47:43.878821 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 19 00:47:43.878821 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 19 00:47:43.878821 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 19 00:47:43.878821 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 19 00:47:43.878821 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 19 00:47:43.878821 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 0da74d88-6672-4df0-b665-b9eec2d073c5 could not be found. Oct 19 00:47:43.878821 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:47:43.878821 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 19 00:47:43.878821 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:47:43.878821 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 19 00:47:43.878821 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 19 00:47:43.878821 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 19 00:47:43.878821 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 19 00:47:43.878821 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 19 00:47:43.878821 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 19 00:47:43.880161 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 19 00:47:43.880161 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 19 00:47:43.880161 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 19 00:47:43.880161 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 19 00:47:43.880161 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 19 00:47:43.880161 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 19 00:47:43.880161 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 19 00:47:43.880161 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 0da74d88-6672-4df0-b665-b9eec2d073c5 could not be found. Oct 19 00:47:43.880161 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:47:43.880161 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 19 00:47:43.880161 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:47:43.880161 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 19 00:47:43.880161 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 19 00:47:43.880161 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 19 00:47:43.880161 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 19 00:47:43.880161 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 19 00:47:43.880161 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 0da74d88-6672-4df0-b665-b9eec2d073c5 could not be found. Oct 19 00:47:43.880161 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:47:43.881645 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-471c591a-5b2d-498c-98a9-3cc5df53e2b1 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Returning 404 to user: Instance 0da74d88-6672-4df0-b665-b9eec2d073c5 could not be found. {{(pid=125641) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 19 00:47:43.881645 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-471c591a-5b2d-498c-98a9-3cc5df53e2b1 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] 10.222.0.26 "GET /compute/v2.1/servers/0da74d88-6672-4df0-b665-b9eec2d073c5" status: 404 len: 111 microversion: 2.1 time: 0.053442 Oct 19 00:47:43.886524 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 683/798] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:47:43 2022] GET /compute/v2.1/servers/0da74d88-6672-4df0-b665-b9eec2d073c5 => generated 111 bytes in 60 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 19 00:47:44.161149 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-f07ed169-c266-40bb-b05a-83f1c472dcde tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:47:44.162324 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-f07ed169-c266-40bb-b05a-83f1c472dcde tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] [instance: b38d4219-8912-4fd9-9d02-7848016f7954] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:47:44.176059 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-f07ed169-c266-40bb-b05a-83f1c472dcde tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:47:44.177377 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-f07ed169-c266-40bb-b05a-83f1c472dcde tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:47:44.180018 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-f07ed169-c266-40bb-b05a-83f1c472dcde tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:47:44.181942 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-f07ed169-c266-40bb-b05a-83f1c472dcde tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:47:44.183546 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-f07ed169-c266-40bb-b05a-83f1c472dcde tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:47:44.185783 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-f07ed169-c266-40bb-b05a-83f1c472dcde tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:47:44.458164 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-f07ed169-c266-40bb-b05a-83f1c472dcde tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a48ed2d9ded741ca91afc1fbbd9c608f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '82d7723ef95b4c3faeac9835eeb233a5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:47:44.468841 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-f07ed169-c266-40bb-b05a-83f1c472dcde tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] 10.222.0.26 "GET /compute/v2.1/servers/b38d4219-8912-4fd9-9d02-7848016f7954" status: 200 len: 1725 microversion: 2.1 time: 0.315266 Oct 19 00:47:44.469692 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 684/799] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:47:44 2022] GET /compute/v2.1/servers/b38d4219-8912-4fd9-9d02-7848016f7954 => generated 1725 bytes in 317 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 19 00:47:45.098950 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-d650fbef-2e67-4092-8d49-3d18dc50da8f tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:47:45.100111 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-d650fbef-2e67-4092-8d49-3d18dc50da8f tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] [instance: d8ff0e88-597f-4db1-b5f4-852dd33c566d] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:47:45.113345 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d650fbef-2e67-4092-8d49-3d18dc50da8f tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:47:45.114356 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d650fbef-2e67-4092-8d49-3d18dc50da8f tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:47:45.115319 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d650fbef-2e67-4092-8d49-3d18dc50da8f tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:47:45.116711 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d650fbef-2e67-4092-8d49-3d18dc50da8f tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:47:45.117639 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d650fbef-2e67-4092-8d49-3d18dc50da8f tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:47:45.118535 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d650fbef-2e67-4092-8d49-3d18dc50da8f tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:47:45.146000 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.wsgi [None req-d650fbef-2e67-4092-8d49-3d18dc50da8f tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] HTTP exception thrown: Instance d8ff0e88-597f-4db1-b5f4-852dd33c566d could not be found. Oct 19 00:47:45.149111 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-d650fbef-2e67-4092-8d49-3d18dc50da8f tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Request method failure captured: Oct 19 00:47:45.149111 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: request: DELETE /compute/v2.1/servers/d8ff0e88-597f-4db1-b5f4-852dd33c566d HTTP/1.1 Oct 19 00:47:45.149111 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Accept: application/json Oct 19 00:47:45.149111 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Accept-Encoding: identity Oct 19 00:47:45.149111 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Connection: close Oct 19 00:47:45.149111 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Content-Length: 0 Oct 19 00:47:45.149111 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Content-Type: application/json Oct 19 00:47:45.149111 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Host: 10.222.0.26 Oct 19 00:47:45.149111 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Openstack-System-Scope: None Oct 19 00:47:45.149111 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: User-Agent: python-urllib3/1.26.12 Oct 19 00:47:45.149111 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Auth-Token: gAAAAABjT0eJXd9ar1txWHw929SSSa4BnzN-f-jxcCDsiqb5Qu6eweguqGUJXghFM8xaTx1ciC3SJFoac87bZHlU5VANYjjzV6ICQChedig2cB1RmNul9h3AYlunCOiAit1Pl73UlpO5HCAxVXGmz0N3G59VsZacZSMYF6RytJzlde2rg7lCpks Oct 19 00:47:45.149111 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Domain-Id: None Oct 19 00:47:45.149111 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Domain-Name: None Oct 19 00:47:45.149111 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Identity-Status: Confirmed Oct 19 00:47:45.149111 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Is-Admin-Project: True Oct 19 00:47:45.149111 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Project-Domain-Id: default Oct 19 00:47:45.149111 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Project-Domain-Name: Default Oct 19 00:47:45.149111 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Project-Id: 0b35c2038f6b4d0a9012b004f1416c00 Oct 19 00:47:45.149111 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Project-Name: tempest-AttachVolumeNegativeTest-1282551484 Oct 19 00:47:45.149111 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Role: member,reader Oct 19 00:47:45.149111 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Roles: member,reader Oct 19 00:47:45.151663 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/0b35c2038f6b4d0a9012b004f1416c00"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/0b35c2038f6b4d0a9012b004f1416c00"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/0b35c2038f6b4d0a9012b004f1416c00"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_0b35c2038f6b4d0a9012b004f1416c00"}]}] Oct 19 00:47:45.151663 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Tenant: tempest-AttachVolumeNegativeTest-1282551484 Oct 19 00:47:45.151663 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Tenant-Id: 0b35c2038f6b4d0a9012b004f1416c00 Oct 19 00:47:45.151663 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1282551484 Oct 19 00:47:45.151663 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User: tempest-AttachVolumeNegativeTest-1282551484-project Oct 19 00:47:45.151663 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User-Domain-Id: default Oct 19 00:47:45.151663 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User-Domain-Name: Default Oct 19 00:47:45.151663 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User-Id: 6f8725c4d05b4dd79b05b43fba15980b Oct 19 00:47:45.151663 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User-Name: tempest-AttachVolumeNegativeTest-1282551484-project Oct 19 00:47:45.151663 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: method: > Oct 19 00:47:45.151663 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: exception: Instance d8ff0e88-597f-4db1-b5f4-852dd33c566d could not be found. Oct 19 00:47:45.151663 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: action_args: {'id': 'd8ff0e88-597f-4db1-b5f4-852dd33c566d'} Oct 19 00:47:45.153023 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 19 00:47:45.153023 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 19 00:47:45.153023 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 19 00:47:45.153023 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 19 00:47:45.153023 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 19 00:47:45.153023 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 19 00:47:45.153023 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 19 00:47:45.153023 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 19 00:47:45.153023 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 19 00:47:45.153023 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 19 00:47:45.153023 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance d8ff0e88-597f-4db1-b5f4-852dd33c566d could not be found. Oct 19 00:47:45.153023 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:47:45.153023 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 19 00:47:45.153023 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:47:45.153023 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 19 00:47:45.153023 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 19 00:47:45.153023 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 19 00:47:45.153023 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 19 00:47:45.153023 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 19 00:47:45.153023 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 19 00:47:45.155858 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 19 00:47:45.155858 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Oct 19 00:47:45.155858 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Oct 19 00:47:45.155858 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Oct 19 00:47:45.155858 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Oct 19 00:47:45.155858 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 19 00:47:45.155858 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 19 00:47:45.155858 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 19 00:47:45.155858 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 19 00:47:45.155858 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance d8ff0e88-597f-4db1-b5f4-852dd33c566d could not be found. Oct 19 00:47:45.155858 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:47:45.155858 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 19 00:47:45.155858 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:47:45.155858 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 19 00:47:45.155858 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 19 00:47:45.155858 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 19 00:47:45.155858 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 19 00:47:45.155858 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 19 00:47:45.155858 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance d8ff0e88-597f-4db1-b5f4-852dd33c566d could not be found. Oct 19 00:47:45.155858 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:47:45.158554 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-d650fbef-2e67-4092-8d49-3d18dc50da8f tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Returning 404 to user: Instance d8ff0e88-597f-4db1-b5f4-852dd33c566d could not be found. {{(pid=125641) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 19 00:47:45.158554 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-d650fbef-2e67-4092-8d49-3d18dc50da8f tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] 10.222.0.26 "DELETE /compute/v2.1/servers/d8ff0e88-597f-4db1-b5f4-852dd33c566d" status: 404 len: 111 microversion: 2.1 time: 0.061028 Oct 19 00:47:45.158554 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 685/800] 10.222.0.26 () {58 vars in 1239 bytes} [Wed Oct 19 00:47:45 2022] DELETE /compute/v2.1/servers/d8ff0e88-597f-4db1-b5f4-852dd33c566d => generated 111 bytes in 64 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Oct 19 00:47:45.164773 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-9f844e4d-bf50-421a-b24b-169826cc32ee tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:47:45.165426 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-9f844e4d-bf50-421a-b24b-169826cc32ee tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] [instance: d8ff0e88-597f-4db1-b5f4-852dd33c566d] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:47:45.172553 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-9f844e4d-bf50-421a-b24b-169826cc32ee tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:47:45.173046 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-9f844e4d-bf50-421a-b24b-169826cc32ee tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:47:45.173481 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-9f844e4d-bf50-421a-b24b-169826cc32ee tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:47:45.174202 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-9f844e4d-bf50-421a-b24b-169826cc32ee tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:47:45.174707 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-9f844e4d-bf50-421a-b24b-169826cc32ee tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:47:45.175834 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-9f844e4d-bf50-421a-b24b-169826cc32ee tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:47:45.192656 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.wsgi [None req-9f844e4d-bf50-421a-b24b-169826cc32ee tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] HTTP exception thrown: Instance d8ff0e88-597f-4db1-b5f4-852dd33c566d could not be found. Oct 19 00:47:45.194262 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-9f844e4d-bf50-421a-b24b-169826cc32ee tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Request method failure captured: Oct 19 00:47:45.194262 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: request: GET /compute/v2.1/servers/d8ff0e88-597f-4db1-b5f4-852dd33c566d HTTP/1.1 Oct 19 00:47:45.194262 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Accept: application/json Oct 19 00:47:45.194262 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Accept-Encoding: identity Oct 19 00:47:45.194262 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Connection: close Oct 19 00:47:45.194262 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Content-Length: 0 Oct 19 00:47:45.194262 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Content-Type: application/json Oct 19 00:47:45.194262 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Host: 10.222.0.26 Oct 19 00:47:45.194262 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Openstack-System-Scope: None Oct 19 00:47:45.194262 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: User-Agent: python-urllib3/1.26.12 Oct 19 00:47:45.194262 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Auth-Token: gAAAAABjT0eJXd9ar1txWHw929SSSa4BnzN-f-jxcCDsiqb5Qu6eweguqGUJXghFM8xaTx1ciC3SJFoac87bZHlU5VANYjjzV6ICQChedig2cB1RmNul9h3AYlunCOiAit1Pl73UlpO5HCAxVXGmz0N3G59VsZacZSMYF6RytJzlde2rg7lCpks Oct 19 00:47:45.194262 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Domain-Id: None Oct 19 00:47:45.194262 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Domain-Name: None Oct 19 00:47:45.194262 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Identity-Status: Confirmed Oct 19 00:47:45.194262 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Is-Admin-Project: True Oct 19 00:47:45.194262 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Project-Domain-Id: default Oct 19 00:47:45.194262 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Project-Domain-Name: Default Oct 19 00:47:45.194262 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Project-Id: 0b35c2038f6b4d0a9012b004f1416c00 Oct 19 00:47:45.194262 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Project-Name: tempest-AttachVolumeNegativeTest-1282551484 Oct 19 00:47:45.194262 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Role: member,reader Oct 19 00:47:45.194262 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Roles: member,reader Oct 19 00:47:45.196793 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/0b35c2038f6b4d0a9012b004f1416c00"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/0b35c2038f6b4d0a9012b004f1416c00"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/0b35c2038f6b4d0a9012b004f1416c00"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_0b35c2038f6b4d0a9012b004f1416c00"}]}] Oct 19 00:47:45.196793 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Tenant: tempest-AttachVolumeNegativeTest-1282551484 Oct 19 00:47:45.196793 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Tenant-Id: 0b35c2038f6b4d0a9012b004f1416c00 Oct 19 00:47:45.196793 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1282551484 Oct 19 00:47:45.196793 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User: tempest-AttachVolumeNegativeTest-1282551484-project Oct 19 00:47:45.196793 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User-Domain-Id: default Oct 19 00:47:45.196793 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User-Domain-Name: Default Oct 19 00:47:45.196793 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User-Id: 6f8725c4d05b4dd79b05b43fba15980b Oct 19 00:47:45.196793 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User-Name: tempest-AttachVolumeNegativeTest-1282551484-project Oct 19 00:47:45.196793 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: method: > Oct 19 00:47:45.196793 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: exception: Instance d8ff0e88-597f-4db1-b5f4-852dd33c566d could not be found. Oct 19 00:47:45.196793 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: action_args: {'id': 'd8ff0e88-597f-4db1-b5f4-852dd33c566d'} Oct 19 00:47:45.198199 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 19 00:47:45.198199 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 19 00:47:45.198199 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 19 00:47:45.198199 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 19 00:47:45.198199 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 19 00:47:45.198199 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 19 00:47:45.198199 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 19 00:47:45.198199 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 19 00:47:45.198199 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 19 00:47:45.198199 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 19 00:47:45.198199 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance d8ff0e88-597f-4db1-b5f4-852dd33c566d could not be found. Oct 19 00:47:45.198199 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:47:45.198199 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 19 00:47:45.198199 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:47:45.198199 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 19 00:47:45.198199 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 19 00:47:45.198199 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 19 00:47:45.198199 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 19 00:47:45.198199 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 19 00:47:45.198199 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 19 00:47:45.200515 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 19 00:47:45.200515 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 19 00:47:45.200515 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 19 00:47:45.200515 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 19 00:47:45.200515 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 19 00:47:45.200515 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 19 00:47:45.200515 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 19 00:47:45.200515 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance d8ff0e88-597f-4db1-b5f4-852dd33c566d could not be found. Oct 19 00:47:45.200515 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:47:45.200515 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 19 00:47:45.200515 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:47:45.200515 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 19 00:47:45.200515 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 19 00:47:45.200515 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 19 00:47:45.200515 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 19 00:47:45.200515 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 19 00:47:45.200515 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance d8ff0e88-597f-4db1-b5f4-852dd33c566d could not be found. Oct 19 00:47:45.200515 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:47:45.203442 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-9f844e4d-bf50-421a-b24b-169826cc32ee tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Returning 404 to user: Instance d8ff0e88-597f-4db1-b5f4-852dd33c566d could not be found. {{(pid=125641) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 19 00:47:45.203442 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-9f844e4d-bf50-421a-b24b-169826cc32ee tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] 10.222.0.26 "GET /compute/v2.1/servers/d8ff0e88-597f-4db1-b5f4-852dd33c566d" status: 404 len: 111 microversion: 2.1 time: 0.036785 Oct 19 00:47:45.203442 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 686/801] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:47:45 2022] GET /compute/v2.1/servers/d8ff0e88-597f-4db1-b5f4-852dd33c566d => generated 111 bytes in 38 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Oct 19 00:47:45.206961 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-447ff2d6-a278-4e8d-89dd-9654765a4eee tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:47:45.207625 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-447ff2d6-a278-4e8d-89dd-9654765a4eee tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] [instance: 25d64f9f-18a6-4b6a-92b1-3e8e7d017ab8] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:47:45.211250 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG dbcounter [-] [125641] Writing DB stats nova_api:SELECT=98,nova_api:UPDATE=1,nova_api:DELETE=1 {{(pid=125641) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 00:47:45.215928 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-447ff2d6-a278-4e8d-89dd-9654765a4eee tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:47:45.216427 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-447ff2d6-a278-4e8d-89dd-9654765a4eee tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:47:45.216955 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-447ff2d6-a278-4e8d-89dd-9654765a4eee tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:47:45.217694 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-447ff2d6-a278-4e8d-89dd-9654765a4eee tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:47:45.218267 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-447ff2d6-a278-4e8d-89dd-9654765a4eee tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:47:45.218779 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-447ff2d6-a278-4e8d-89dd-9654765a4eee tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:47:45.239589 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.wsgi [None req-447ff2d6-a278-4e8d-89dd-9654765a4eee tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] HTTP exception thrown: Instance 25d64f9f-18a6-4b6a-92b1-3e8e7d017ab8 could not be found. Oct 19 00:47:45.241280 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-447ff2d6-a278-4e8d-89dd-9654765a4eee tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Request method failure captured: Oct 19 00:47:45.241280 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: request: DELETE /compute/v2.1/servers/25d64f9f-18a6-4b6a-92b1-3e8e7d017ab8 HTTP/1.1 Oct 19 00:47:45.241280 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Accept: application/json Oct 19 00:47:45.241280 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Accept-Encoding: identity Oct 19 00:47:45.241280 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Connection: close Oct 19 00:47:45.241280 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Content-Length: 0 Oct 19 00:47:45.241280 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Content-Type: application/json Oct 19 00:47:45.241280 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Host: 10.222.0.26 Oct 19 00:47:45.241280 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Openstack-System-Scope: None Oct 19 00:47:45.241280 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: User-Agent: python-urllib3/1.26.12 Oct 19 00:47:45.241280 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Auth-Token: gAAAAABjT0eJXd9ar1txWHw929SSSa4BnzN-f-jxcCDsiqb5Qu6eweguqGUJXghFM8xaTx1ciC3SJFoac87bZHlU5VANYjjzV6ICQChedig2cB1RmNul9h3AYlunCOiAit1Pl73UlpO5HCAxVXGmz0N3G59VsZacZSMYF6RytJzlde2rg7lCpks Oct 19 00:47:45.241280 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Domain-Id: None Oct 19 00:47:45.241280 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Domain-Name: None Oct 19 00:47:45.241280 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Identity-Status: Confirmed Oct 19 00:47:45.241280 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Is-Admin-Project: True Oct 19 00:47:45.241280 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Project-Domain-Id: default Oct 19 00:47:45.241280 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Project-Domain-Name: Default Oct 19 00:47:45.241280 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Project-Id: 0b35c2038f6b4d0a9012b004f1416c00 Oct 19 00:47:45.241280 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Project-Name: tempest-AttachVolumeNegativeTest-1282551484 Oct 19 00:47:45.241280 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Role: member,reader Oct 19 00:47:45.241280 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Roles: member,reader Oct 19 00:47:45.243764 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/0b35c2038f6b4d0a9012b004f1416c00"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/0b35c2038f6b4d0a9012b004f1416c00"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/0b35c2038f6b4d0a9012b004f1416c00"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_0b35c2038f6b4d0a9012b004f1416c00"}]}] Oct 19 00:47:45.243764 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Tenant: tempest-AttachVolumeNegativeTest-1282551484 Oct 19 00:47:45.243764 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Tenant-Id: 0b35c2038f6b4d0a9012b004f1416c00 Oct 19 00:47:45.243764 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1282551484 Oct 19 00:47:45.243764 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User: tempest-AttachVolumeNegativeTest-1282551484-project Oct 19 00:47:45.243764 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User-Domain-Id: default Oct 19 00:47:45.243764 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User-Domain-Name: Default Oct 19 00:47:45.243764 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User-Id: 6f8725c4d05b4dd79b05b43fba15980b Oct 19 00:47:45.243764 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User-Name: tempest-AttachVolumeNegativeTest-1282551484-project Oct 19 00:47:45.243764 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: method: > Oct 19 00:47:45.243764 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: exception: Instance 25d64f9f-18a6-4b6a-92b1-3e8e7d017ab8 could not be found. Oct 19 00:47:45.243764 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: action_args: {'id': '25d64f9f-18a6-4b6a-92b1-3e8e7d017ab8'} Oct 19 00:47:45.245409 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 19 00:47:45.245409 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 19 00:47:45.245409 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 19 00:47:45.245409 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 19 00:47:45.245409 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 19 00:47:45.245409 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 19 00:47:45.245409 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 19 00:47:45.245409 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 19 00:47:45.245409 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 19 00:47:45.245409 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 19 00:47:45.245409 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 25d64f9f-18a6-4b6a-92b1-3e8e7d017ab8 could not be found. Oct 19 00:47:45.245409 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:47:45.245409 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 19 00:47:45.245409 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:47:45.245409 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 19 00:47:45.245409 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 19 00:47:45.245409 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 19 00:47:45.245409 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 19 00:47:45.245409 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 19 00:47:45.245409 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 19 00:47:45.248110 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 19 00:47:45.248110 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Oct 19 00:47:45.248110 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Oct 19 00:47:45.248110 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Oct 19 00:47:45.248110 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Oct 19 00:47:45.248110 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 19 00:47:45.248110 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 19 00:47:45.248110 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 19 00:47:45.248110 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 19 00:47:45.248110 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 25d64f9f-18a6-4b6a-92b1-3e8e7d017ab8 could not be found. Oct 19 00:47:45.248110 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:47:45.248110 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 19 00:47:45.248110 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:47:45.248110 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 19 00:47:45.248110 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 19 00:47:45.248110 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 19 00:47:45.248110 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 19 00:47:45.248110 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 19 00:47:45.248110 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 25d64f9f-18a6-4b6a-92b1-3e8e7d017ab8 could not be found. Oct 19 00:47:45.248110 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:47:45.250323 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-447ff2d6-a278-4e8d-89dd-9654765a4eee tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Returning 404 to user: Instance 25d64f9f-18a6-4b6a-92b1-3e8e7d017ab8 could not be found. {{(pid=125641) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 19 00:47:45.250323 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-447ff2d6-a278-4e8d-89dd-9654765a4eee tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] 10.222.0.26 "DELETE /compute/v2.1/servers/25d64f9f-18a6-4b6a-92b1-3e8e7d017ab8" status: 404 len: 111 microversion: 2.1 time: 0.040089 Oct 19 00:47:45.250323 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 687/802] 10.222.0.26 () {58 vars in 1239 bytes} [Wed Oct 19 00:47:45 2022] DELETE /compute/v2.1/servers/25d64f9f-18a6-4b6a-92b1-3e8e7d017ab8 => generated 111 bytes in 41 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Oct 19 00:47:45.254555 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-226bfeaa-1330-4410-a575-44d54f8c4ea3 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:47:45.255408 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-226bfeaa-1330-4410-a575-44d54f8c4ea3 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] [instance: 25d64f9f-18a6-4b6a-92b1-3e8e7d017ab8] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:47:45.265754 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-226bfeaa-1330-4410-a575-44d54f8c4ea3 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:47:45.266476 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-226bfeaa-1330-4410-a575-44d54f8c4ea3 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:47:45.267363 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-226bfeaa-1330-4410-a575-44d54f8c4ea3 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:47:45.268596 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-226bfeaa-1330-4410-a575-44d54f8c4ea3 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:47:45.269710 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-226bfeaa-1330-4410-a575-44d54f8c4ea3 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:47:45.270165 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-226bfeaa-1330-4410-a575-44d54f8c4ea3 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:47:45.294794 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.wsgi [None req-226bfeaa-1330-4410-a575-44d54f8c4ea3 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] HTTP exception thrown: Instance 25d64f9f-18a6-4b6a-92b1-3e8e7d017ab8 could not be found. Oct 19 00:47:45.296465 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-226bfeaa-1330-4410-a575-44d54f8c4ea3 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Request method failure captured: Oct 19 00:47:45.296465 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: request: GET /compute/v2.1/servers/25d64f9f-18a6-4b6a-92b1-3e8e7d017ab8 HTTP/1.1 Oct 19 00:47:45.296465 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Accept: application/json Oct 19 00:47:45.296465 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Accept-Encoding: identity Oct 19 00:47:45.296465 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Connection: close Oct 19 00:47:45.296465 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Content-Length: 0 Oct 19 00:47:45.296465 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Content-Type: application/json Oct 19 00:47:45.296465 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Host: 10.222.0.26 Oct 19 00:47:45.296465 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Openstack-System-Scope: None Oct 19 00:47:45.296465 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: User-Agent: python-urllib3/1.26.12 Oct 19 00:47:45.296465 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Auth-Token: gAAAAABjT0eJXd9ar1txWHw929SSSa4BnzN-f-jxcCDsiqb5Qu6eweguqGUJXghFM8xaTx1ciC3SJFoac87bZHlU5VANYjjzV6ICQChedig2cB1RmNul9h3AYlunCOiAit1Pl73UlpO5HCAxVXGmz0N3G59VsZacZSMYF6RytJzlde2rg7lCpks Oct 19 00:47:45.296465 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Domain-Id: None Oct 19 00:47:45.296465 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Domain-Name: None Oct 19 00:47:45.296465 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Identity-Status: Confirmed Oct 19 00:47:45.296465 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Is-Admin-Project: True Oct 19 00:47:45.296465 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Project-Domain-Id: default Oct 19 00:47:45.296465 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Project-Domain-Name: Default Oct 19 00:47:45.296465 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Project-Id: 0b35c2038f6b4d0a9012b004f1416c00 Oct 19 00:47:45.296465 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Project-Name: tempest-AttachVolumeNegativeTest-1282551484 Oct 19 00:47:45.296465 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Role: member,reader Oct 19 00:47:45.296465 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Roles: member,reader Oct 19 00:47:45.298979 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/0b35c2038f6b4d0a9012b004f1416c00"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/0b35c2038f6b4d0a9012b004f1416c00"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/0b35c2038f6b4d0a9012b004f1416c00"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_0b35c2038f6b4d0a9012b004f1416c00"}]}] Oct 19 00:47:45.298979 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Tenant: tempest-AttachVolumeNegativeTest-1282551484 Oct 19 00:47:45.298979 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Tenant-Id: 0b35c2038f6b4d0a9012b004f1416c00 Oct 19 00:47:45.298979 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1282551484 Oct 19 00:47:45.298979 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User: tempest-AttachVolumeNegativeTest-1282551484-project Oct 19 00:47:45.298979 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User-Domain-Id: default Oct 19 00:47:45.298979 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User-Domain-Name: Default Oct 19 00:47:45.298979 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User-Id: 6f8725c4d05b4dd79b05b43fba15980b Oct 19 00:47:45.298979 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User-Name: tempest-AttachVolumeNegativeTest-1282551484-project Oct 19 00:47:45.298979 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: method: > Oct 19 00:47:45.298979 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: exception: Instance 25d64f9f-18a6-4b6a-92b1-3e8e7d017ab8 could not be found. Oct 19 00:47:45.298979 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: action_args: {'id': '25d64f9f-18a6-4b6a-92b1-3e8e7d017ab8'} Oct 19 00:47:45.301882 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 19 00:47:45.301882 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 19 00:47:45.301882 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 19 00:47:45.301882 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 19 00:47:45.301882 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 19 00:47:45.301882 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 19 00:47:45.301882 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 19 00:47:45.301882 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 19 00:47:45.301882 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 19 00:47:45.301882 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 19 00:47:45.301882 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 25d64f9f-18a6-4b6a-92b1-3e8e7d017ab8 could not be found. Oct 19 00:47:45.301882 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:47:45.301882 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 19 00:47:45.301882 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:47:45.301882 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 19 00:47:45.301882 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 19 00:47:45.301882 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 19 00:47:45.301882 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 19 00:47:45.301882 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 19 00:47:45.301882 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 19 00:47:45.304741 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 19 00:47:45.304741 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 19 00:47:45.304741 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 19 00:47:45.304741 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 19 00:47:45.304741 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 19 00:47:45.304741 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 19 00:47:45.304741 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 19 00:47:45.304741 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 25d64f9f-18a6-4b6a-92b1-3e8e7d017ab8 could not be found. Oct 19 00:47:45.304741 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:47:45.304741 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 19 00:47:45.304741 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:47:45.304741 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 19 00:47:45.304741 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 19 00:47:45.304741 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 19 00:47:45.304741 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 19 00:47:45.304741 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 19 00:47:45.304741 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 25d64f9f-18a6-4b6a-92b1-3e8e7d017ab8 could not be found. Oct 19 00:47:45.304741 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:47:45.307181 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-226bfeaa-1330-4410-a575-44d54f8c4ea3 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Returning 404 to user: Instance 25d64f9f-18a6-4b6a-92b1-3e8e7d017ab8 could not be found. {{(pid=125641) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 19 00:47:45.307181 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-226bfeaa-1330-4410-a575-44d54f8c4ea3 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] 10.222.0.26 "GET /compute/v2.1/servers/25d64f9f-18a6-4b6a-92b1-3e8e7d017ab8" status: 404 len: 111 microversion: 2.1 time: 0.048811 Oct 19 00:47:45.307181 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 688/803] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:47:45 2022] GET /compute/v2.1/servers/25d64f9f-18a6-4b6a-92b1-3e8e7d017ab8 => generated 111 bytes in 50 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Oct 19 00:47:45.505959 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-f65da35c-9117-48a1-8f49-30b59ee1920a tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:47:45.506723 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-f65da35c-9117-48a1-8f49-30b59ee1920a tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] [instance: b38d4219-8912-4fd9-9d02-7848016f7954] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:47:45.513353 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-f65da35c-9117-48a1-8f49-30b59ee1920a tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:47:45.513851 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-f65da35c-9117-48a1-8f49-30b59ee1920a tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:47:45.514294 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-f65da35c-9117-48a1-8f49-30b59ee1920a tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:47:45.515052 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-f65da35c-9117-48a1-8f49-30b59ee1920a tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:47:45.515514 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-f65da35c-9117-48a1-8f49-30b59ee1920a tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:47:45.515942 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-f65da35c-9117-48a1-8f49-30b59ee1920a tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:47:45.675026 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-f65da35c-9117-48a1-8f49-30b59ee1920a tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a48ed2d9ded741ca91afc1fbbd9c608f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '82d7723ef95b4c3faeac9835eeb233a5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:47:45.684684 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-f65da35c-9117-48a1-8f49-30b59ee1920a tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] 10.222.0.26 "GET /compute/v2.1/servers/b38d4219-8912-4fd9-9d02-7848016f7954" status: 200 len: 1702 microversion: 2.1 time: 0.183635 Oct 19 00:47:45.685304 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 689/804] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:47:45 2022] GET /compute/v2.1/servers/b38d4219-8912-4fd9-9d02-7848016f7954 => generated 1702 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:47:45.712993 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-a35ef613-8be7-4258-8697-4e6897e74b11 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Action: 'action', calling method: >, body: {"os-start": {}} {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 19 00:47:45.722628 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-a35ef613-8be7-4258-8697-4e6897e74b11 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:47:45.723979 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-a35ef613-8be7-4258-8697-4e6897e74b11 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:47:45.724552 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-a35ef613-8be7-4258-8697-4e6897e74b11 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:47:45.751918 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-a35ef613-8be7-4258-8697-4e6897e74b11 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] [instance: b38d4219-8912-4fd9-9d02-7848016f7954] Going to try to start instance {{(pid=125641) start /opt/stack/nova/nova/compute/api.py:2785}} Oct 19 00:47:45.756065 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG dbcounter [-] [125641] Writing DB stats nova_cell1:SELECT=100 {{(pid=125641) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 00:47:45.812506 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-a35ef613-8be7-4258-8697-4e6897e74b11 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] 10.222.0.26 "POST /compute/v2.1/servers/b38d4219-8912-4fd9-9d02-7848016f7954/action" status: 202 len: 0 microversion: 2.1 time: 0.105876 Oct 19 00:47:45.813129 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 690/805] 10.222.0.26 () {60 vars in 1278 bytes} [Wed Oct 19 00:47:45 2022] POST /compute/v2.1/servers/b38d4219-8912-4fd9-9d02-7848016f7954/action => generated 0 bytes in 108 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Oct 19 00:47:45.822684 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-09668f83-7a34-45e6-a604-6c7d457263b4 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:47:45.823423 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-09668f83-7a34-45e6-a604-6c7d457263b4 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] [instance: b38d4219-8912-4fd9-9d02-7848016f7954] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:47:45.830375 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-09668f83-7a34-45e6-a604-6c7d457263b4 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:47:45.830912 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-09668f83-7a34-45e6-a604-6c7d457263b4 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:47:45.831890 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-09668f83-7a34-45e6-a604-6c7d457263b4 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:47:45.832989 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-09668f83-7a34-45e6-a604-6c7d457263b4 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:47:45.833506 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-09668f83-7a34-45e6-a604-6c7d457263b4 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:47:45.833978 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-09668f83-7a34-45e6-a604-6c7d457263b4 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:47:46.021978 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-09668f83-7a34-45e6-a604-6c7d457263b4 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a48ed2d9ded741ca91afc1fbbd9c608f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '82d7723ef95b4c3faeac9835eeb233a5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:47:46.032717 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-09668f83-7a34-45e6-a604-6c7d457263b4 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] 10.222.0.26 "GET /compute/v2.1/servers/b38d4219-8912-4fd9-9d02-7848016f7954" status: 200 len: 1711 microversion: 2.1 time: 0.214599 Oct 19 00:47:46.033552 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 691/806] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:47:45 2022] GET /compute/v2.1/servers/b38d4219-8912-4fd9-9d02-7848016f7954 => generated 1711 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 19 00:47:46.480857 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-3a8c74d7-3dd1-4797-99c5-da5bc009c768 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:47:46.481929 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-3a8c74d7-3dd1-4797-99c5-da5bc009c768 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] [instance: dc8b8a6e-eab7-4f45-8f5d-916c52eec398] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:47:46.489832 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-3a8c74d7-3dd1-4797-99c5-da5bc009c768 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:47:46.490334 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-3a8c74d7-3dd1-4797-99c5-da5bc009c768 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:47:46.490987 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-3a8c74d7-3dd1-4797-99c5-da5bc009c768 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:47:46.491730 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-3a8c74d7-3dd1-4797-99c5-da5bc009c768 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:47:46.492191 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-3a8c74d7-3dd1-4797-99c5-da5bc009c768 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:47:46.492613 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-3a8c74d7-3dd1-4797-99c5-da5bc009c768 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:47:46.509664 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.wsgi [None req-3a8c74d7-3dd1-4797-99c5-da5bc009c768 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] HTTP exception thrown: Instance dc8b8a6e-eab7-4f45-8f5d-916c52eec398 could not be found. Oct 19 00:47:46.511307 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-3a8c74d7-3dd1-4797-99c5-da5bc009c768 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Request method failure captured: Oct 19 00:47:46.511307 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: request: DELETE /compute/v2.1/servers/dc8b8a6e-eab7-4f45-8f5d-916c52eec398 HTTP/1.1 Oct 19 00:47:46.511307 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Accept: application/json Oct 19 00:47:46.511307 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Accept-Encoding: identity Oct 19 00:47:46.511307 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Connection: close Oct 19 00:47:46.511307 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Content-Length: 0 Oct 19 00:47:46.511307 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Content-Type: application/json Oct 19 00:47:46.511307 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Host: 10.222.0.26 Oct 19 00:47:46.511307 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Openstack-System-Scope: None Oct 19 00:47:46.511307 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: User-Agent: python-urllib3/1.26.12 Oct 19 00:47:46.511307 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Auth-Token: gAAAAABjT0eJXd9ar1txWHw929SSSa4BnzN-f-jxcCDsiqb5Qu6eweguqGUJXghFM8xaTx1ciC3SJFoac87bZHlU5VANYjjzV6ICQChedig2cB1RmNul9h3AYlunCOiAit1Pl73UlpO5HCAxVXGmz0N3G59VsZacZSMYF6RytJzlde2rg7lCpks Oct 19 00:47:46.511307 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Domain-Id: None Oct 19 00:47:46.511307 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Domain-Name: None Oct 19 00:47:46.511307 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Identity-Status: Confirmed Oct 19 00:47:46.511307 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Is-Admin-Project: True Oct 19 00:47:46.511307 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Project-Domain-Id: default Oct 19 00:47:46.511307 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Project-Domain-Name: Default Oct 19 00:47:46.511307 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Project-Id: 0b35c2038f6b4d0a9012b004f1416c00 Oct 19 00:47:46.511307 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Project-Name: tempest-AttachVolumeNegativeTest-1282551484 Oct 19 00:47:46.511307 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Role: member,reader Oct 19 00:47:46.511307 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Roles: member,reader Oct 19 00:47:46.514064 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/0b35c2038f6b4d0a9012b004f1416c00"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/0b35c2038f6b4d0a9012b004f1416c00"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/0b35c2038f6b4d0a9012b004f1416c00"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_0b35c2038f6b4d0a9012b004f1416c00"}]}] Oct 19 00:47:46.514064 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Tenant: tempest-AttachVolumeNegativeTest-1282551484 Oct 19 00:47:46.514064 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Tenant-Id: 0b35c2038f6b4d0a9012b004f1416c00 Oct 19 00:47:46.514064 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1282551484 Oct 19 00:47:46.514064 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User: tempest-AttachVolumeNegativeTest-1282551484-project Oct 19 00:47:46.514064 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User-Domain-Id: default Oct 19 00:47:46.514064 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User-Domain-Name: Default Oct 19 00:47:46.514064 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User-Id: 6f8725c4d05b4dd79b05b43fba15980b Oct 19 00:47:46.514064 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User-Name: tempest-AttachVolumeNegativeTest-1282551484-project Oct 19 00:47:46.514064 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: method: > Oct 19 00:47:46.514064 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: exception: Instance dc8b8a6e-eab7-4f45-8f5d-916c52eec398 could not be found. Oct 19 00:47:46.514064 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: action_args: {'id': 'dc8b8a6e-eab7-4f45-8f5d-916c52eec398'} Oct 19 00:47:46.515962 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 19 00:47:46.515962 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 19 00:47:46.515962 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 19 00:47:46.515962 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 19 00:47:46.515962 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 19 00:47:46.515962 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 19 00:47:46.515962 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 19 00:47:46.515962 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 19 00:47:46.515962 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 19 00:47:46.515962 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 19 00:47:46.515962 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance dc8b8a6e-eab7-4f45-8f5d-916c52eec398 could not be found. Oct 19 00:47:46.515962 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:47:46.515962 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 19 00:47:46.515962 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:47:46.515962 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 19 00:47:46.515962 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 19 00:47:46.515962 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 19 00:47:46.515962 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 19 00:47:46.515962 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 19 00:47:46.515962 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 19 00:47:46.518555 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 19 00:47:46.518555 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Oct 19 00:47:46.518555 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Oct 19 00:47:46.518555 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Oct 19 00:47:46.518555 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Oct 19 00:47:46.518555 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 19 00:47:46.518555 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 19 00:47:46.518555 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 19 00:47:46.518555 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 19 00:47:46.518555 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance dc8b8a6e-eab7-4f45-8f5d-916c52eec398 could not be found. Oct 19 00:47:46.518555 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:47:46.518555 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 19 00:47:46.518555 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:47:46.518555 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 19 00:47:46.518555 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 19 00:47:46.518555 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 19 00:47:46.518555 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 19 00:47:46.518555 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 19 00:47:46.518555 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance dc8b8a6e-eab7-4f45-8f5d-916c52eec398 could not be found. Oct 19 00:47:46.518555 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:47:46.521212 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-3a8c74d7-3dd1-4797-99c5-da5bc009c768 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Returning 404 to user: Instance dc8b8a6e-eab7-4f45-8f5d-916c52eec398 could not be found. {{(pid=125641) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 19 00:47:46.521212 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-3a8c74d7-3dd1-4797-99c5-da5bc009c768 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] 10.222.0.26 "DELETE /compute/v2.1/servers/dc8b8a6e-eab7-4f45-8f5d-916c52eec398" status: 404 len: 111 microversion: 2.1 time: 0.036064 Oct 19 00:47:46.521212 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 692/807] 10.222.0.26 () {58 vars in 1239 bytes} [Wed Oct 19 00:47:46 2022] DELETE /compute/v2.1/servers/dc8b8a6e-eab7-4f45-8f5d-916c52eec398 => generated 111 bytes in 37 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Oct 19 00:47:46.522295 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-f090d66b-7378-4303-86d8-045f7a8c8182 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:47:46.522615 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-f090d66b-7378-4303-86d8-045f7a8c8182 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] [instance: dc8b8a6e-eab7-4f45-8f5d-916c52eec398] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:47:46.529209 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-f090d66b-7378-4303-86d8-045f7a8c8182 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:47:46.529702 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-f090d66b-7378-4303-86d8-045f7a8c8182 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:47:46.530171 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-f090d66b-7378-4303-86d8-045f7a8c8182 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:47:46.530915 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-f090d66b-7378-4303-86d8-045f7a8c8182 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:47:46.531376 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-f090d66b-7378-4303-86d8-045f7a8c8182 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:47:46.531801 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-f090d66b-7378-4303-86d8-045f7a8c8182 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:47:46.553651 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.wsgi [None req-f090d66b-7378-4303-86d8-045f7a8c8182 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] HTTP exception thrown: Instance dc8b8a6e-eab7-4f45-8f5d-916c52eec398 could not be found. Oct 19 00:47:46.556968 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-f090d66b-7378-4303-86d8-045f7a8c8182 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Request method failure captured: Oct 19 00:47:46.556968 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: request: GET /compute/v2.1/servers/dc8b8a6e-eab7-4f45-8f5d-916c52eec398 HTTP/1.1 Oct 19 00:47:46.556968 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Accept: application/json Oct 19 00:47:46.556968 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Accept-Encoding: identity Oct 19 00:47:46.556968 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Connection: close Oct 19 00:47:46.556968 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Content-Length: 0 Oct 19 00:47:46.556968 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Content-Type: application/json Oct 19 00:47:46.556968 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Host: 10.222.0.26 Oct 19 00:47:46.556968 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Openstack-System-Scope: None Oct 19 00:47:46.556968 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: User-Agent: python-urllib3/1.26.12 Oct 19 00:47:46.556968 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Auth-Token: gAAAAABjT0eJXd9ar1txWHw929SSSa4BnzN-f-jxcCDsiqb5Qu6eweguqGUJXghFM8xaTx1ciC3SJFoac87bZHlU5VANYjjzV6ICQChedig2cB1RmNul9h3AYlunCOiAit1Pl73UlpO5HCAxVXGmz0N3G59VsZacZSMYF6RytJzlde2rg7lCpks Oct 19 00:47:46.556968 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Domain-Id: None Oct 19 00:47:46.556968 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Domain-Name: None Oct 19 00:47:46.556968 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Identity-Status: Confirmed Oct 19 00:47:46.556968 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Is-Admin-Project: True Oct 19 00:47:46.556968 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Project-Domain-Id: default Oct 19 00:47:46.556968 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Project-Domain-Name: Default Oct 19 00:47:46.556968 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Project-Id: 0b35c2038f6b4d0a9012b004f1416c00 Oct 19 00:47:46.556968 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Project-Name: tempest-AttachVolumeNegativeTest-1282551484 Oct 19 00:47:46.556968 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Role: member,reader Oct 19 00:47:46.556968 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Roles: member,reader Oct 19 00:47:46.559530 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/0b35c2038f6b4d0a9012b004f1416c00"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/0b35c2038f6b4d0a9012b004f1416c00"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/0b35c2038f6b4d0a9012b004f1416c00"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_0b35c2038f6b4d0a9012b004f1416c00"}]}] Oct 19 00:47:46.559530 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Tenant: tempest-AttachVolumeNegativeTest-1282551484 Oct 19 00:47:46.559530 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Tenant-Id: 0b35c2038f6b4d0a9012b004f1416c00 Oct 19 00:47:46.559530 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1282551484 Oct 19 00:47:46.559530 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User: tempest-AttachVolumeNegativeTest-1282551484-project Oct 19 00:47:46.559530 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User-Domain-Id: default Oct 19 00:47:46.559530 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User-Domain-Name: Default Oct 19 00:47:46.559530 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User-Id: 6f8725c4d05b4dd79b05b43fba15980b Oct 19 00:47:46.559530 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User-Name: tempest-AttachVolumeNegativeTest-1282551484-project Oct 19 00:47:46.559530 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: method: > Oct 19 00:47:46.559530 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: exception: Instance dc8b8a6e-eab7-4f45-8f5d-916c52eec398 could not be found. Oct 19 00:47:46.559530 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: action_args: {'id': 'dc8b8a6e-eab7-4f45-8f5d-916c52eec398'} Oct 19 00:47:46.560947 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 19 00:47:46.560947 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 19 00:47:46.560947 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 19 00:47:46.560947 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 19 00:47:46.560947 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 19 00:47:46.560947 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 19 00:47:46.560947 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 19 00:47:46.560947 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 19 00:47:46.560947 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 19 00:47:46.560947 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 19 00:47:46.560947 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance dc8b8a6e-eab7-4f45-8f5d-916c52eec398 could not be found. Oct 19 00:47:46.560947 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:47:46.560947 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 19 00:47:46.560947 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:47:46.560947 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 19 00:47:46.560947 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 19 00:47:46.560947 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 19 00:47:46.560947 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 19 00:47:46.560947 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 19 00:47:46.560947 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 19 00:47:46.565691 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 19 00:47:46.565691 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 19 00:47:46.565691 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 19 00:47:46.565691 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 19 00:47:46.565691 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 19 00:47:46.565691 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 19 00:47:46.565691 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 19 00:47:46.565691 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance dc8b8a6e-eab7-4f45-8f5d-916c52eec398 could not be found. Oct 19 00:47:46.565691 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:47:46.565691 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 19 00:47:46.565691 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:47:46.565691 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 19 00:47:46.565691 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 19 00:47:46.565691 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 19 00:47:46.565691 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 19 00:47:46.565691 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 19 00:47:46.565691 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance dc8b8a6e-eab7-4f45-8f5d-916c52eec398 could not be found. Oct 19 00:47:46.565691 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:47:46.567822 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-f090d66b-7378-4303-86d8-045f7a8c8182 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] Returning 404 to user: Instance dc8b8a6e-eab7-4f45-8f5d-916c52eec398 could not be found. {{(pid=125641) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 19 00:47:46.567822 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-f090d66b-7378-4303-86d8-045f7a8c8182 tempest-AttachVolumeNegativeTest-1282551484 tempest-AttachVolumeNegativeTest-1282551484-project] 10.222.0.26 "GET /compute/v2.1/servers/dc8b8a6e-eab7-4f45-8f5d-916c52eec398" status: 404 len: 111 microversion: 2.1 time: 0.043473 Oct 19 00:47:46.567822 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 693/808] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:47:46 2022] GET /compute/v2.1/servers/dc8b8a6e-eab7-4f45-8f5d-916c52eec398 => generated 111 bytes in 45 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Oct 19 00:47:46.885526 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-eb29b7bd-2905-4108-8a30-5c82cb221b9b tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:47:46.887898 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-eb29b7bd-2905-4108-8a30-5c82cb221b9b tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] [instance: 85efeeae-0648-4a33-bc74-c0f0d25880aa] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:47:46.895258 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-eb29b7bd-2905-4108-8a30-5c82cb221b9b tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:47:46.895779 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-eb29b7bd-2905-4108-8a30-5c82cb221b9b tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:47:46.896280 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-eb29b7bd-2905-4108-8a30-5c82cb221b9b tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:47:46.897034 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-eb29b7bd-2905-4108-8a30-5c82cb221b9b tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:47:46.897508 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-eb29b7bd-2905-4108-8a30-5c82cb221b9b tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:47:46.897941 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-eb29b7bd-2905-4108-8a30-5c82cb221b9b tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:47:46.935849 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-eb29b7bd-2905-4108-8a30-5c82cb221b9b tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] 10.222.0.26 "GET /compute/v2.1/servers/85efeeae-0648-4a33-bc74-c0f0d25880aa/os-volume_attachments" status: 200 len: 25 microversion: 2.20 time: 0.054398 Oct 19 00:47:46.936884 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 694/809] 10.222.0.26 () {60 vars in 1343 bytes} [Wed Oct 19 00:47:46 2022] GET /compute/v2.1/servers/85efeeae-0648-4a33-bc74-c0f0d25880aa/os-volume_attachments => generated 25 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 19 00:47:47.062051 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-fd937c1b-ed0d-4769-b4ae-f97e3f75e524 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:47:47.063428 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-fd937c1b-ed0d-4769-b4ae-f97e3f75e524 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] [instance: b38d4219-8912-4fd9-9d02-7848016f7954] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:47:47.071826 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-fd937c1b-ed0d-4769-b4ae-f97e3f75e524 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:47:47.072492 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-fd937c1b-ed0d-4769-b4ae-f97e3f75e524 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:47:47.073077 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-fd937c1b-ed0d-4769-b4ae-f97e3f75e524 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:47:47.073989 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-fd937c1b-ed0d-4769-b4ae-f97e3f75e524 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:47:47.074576 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-fd937c1b-ed0d-4769-b4ae-f97e3f75e524 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:47:47.075124 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-fd937c1b-ed0d-4769-b4ae-f97e3f75e524 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:47:47.321132 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-fd937c1b-ed0d-4769-b4ae-f97e3f75e524 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a48ed2d9ded741ca91afc1fbbd9c608f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '82d7723ef95b4c3faeac9835eeb233a5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:47:47.335480 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-fd937c1b-ed0d-4769-b4ae-f97e3f75e524 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] 10.222.0.26 "GET /compute/v2.1/servers/b38d4219-8912-4fd9-9d02-7848016f7954" status: 200 len: 1711 microversion: 2.1 time: 0.278351 Oct 19 00:47:47.336595 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 695/810] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:47:47 2022] GET /compute/v2.1/servers/b38d4219-8912-4fd9-9d02-7848016f7954 => generated 1711 bytes in 281 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 19 00:47:47.346011 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-e5c91617-f232-4b58-a9cf-d6853a7fda09 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:47:47.347600 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-e5c91617-f232-4b58-a9cf-d6853a7fda09 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] [instance: 85efeeae-0648-4a33-bc74-c0f0d25880aa] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:47:47.359824 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-e5c91617-f232-4b58-a9cf-d6853a7fda09 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:47:47.361076 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-e5c91617-f232-4b58-a9cf-d6853a7fda09 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:47:47.361664 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-e5c91617-f232-4b58-a9cf-d6853a7fda09 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:47:47.362959 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-e5c91617-f232-4b58-a9cf-d6853a7fda09 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:47:47.363960 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-e5c91617-f232-4b58-a9cf-d6853a7fda09 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:47:47.364752 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-e5c91617-f232-4b58-a9cf-d6853a7fda09 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:47:47.395590 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-e5c91617-f232-4b58-a9cf-d6853a7fda09 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] [instance: 85efeeae-0648-4a33-bc74-c0f0d25880aa] Going to try to terminate instance {{(pid=125641) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 19 00:47:47.499236 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-e5c91617-f232-4b58-a9cf-d6853a7fda09 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] 10.222.0.26 "DELETE /compute/v2.1/servers/85efeeae-0648-4a33-bc74-c0f0d25880aa" status: 204 len: 0 microversion: 2.20 time: 0.160747 Oct 19 00:47:47.500193 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 696/811] 10.222.0.26 () {60 vars in 1280 bytes} [Wed Oct 19 00:47:47 2022] DELETE /compute/v2.1/servers/85efeeae-0648-4a33-bc74-c0f0d25880aa => generated 0 bytes in 163 msecs (HTTP/1.1 204) 9 headers in 365 bytes (1 switches on core 0) Oct 19 00:47:47.511118 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-bfc02f0e-3179-4ef4-bdb7-ec26ff6f82dc tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:47:47.511859 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-bfc02f0e-3179-4ef4-bdb7-ec26ff6f82dc tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] [instance: 85efeeae-0648-4a33-bc74-c0f0d25880aa] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:47:47.519789 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-bfc02f0e-3179-4ef4-bdb7-ec26ff6f82dc tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:47:47.520825 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-bfc02f0e-3179-4ef4-bdb7-ec26ff6f82dc tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:47:47.521317 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-bfc02f0e-3179-4ef4-bdb7-ec26ff6f82dc tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:47:47.522180 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-bfc02f0e-3179-4ef4-bdb7-ec26ff6f82dc tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:47:47.522684 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-bfc02f0e-3179-4ef4-bdb7-ec26ff6f82dc tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:47:47.523126 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-bfc02f0e-3179-4ef4-bdb7-ec26ff6f82dc tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:47:47.777456 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-bfc02f0e-3179-4ef4-bdb7-ec26ff6f82dc tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:47:47.798868 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-bfc02f0e-3179-4ef4-bdb7-ec26ff6f82dc tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:47:47.802757 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-bfc02f0e-3179-4ef4-bdb7-ec26ff6f82dc tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:47:47.805030 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-bfc02f0e-3179-4ef4-bdb7-ec26ff6f82dc tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] 10.222.0.26 "GET /compute/v2.1/servers/85efeeae-0648-4a33-bc74-c0f0d25880aa" status: 200 len: 1723 microversion: 2.20 time: 0.298564 Oct 19 00:47:47.806033 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 697/812] 10.222.0.26 () {60 vars in 1277 bytes} [Wed Oct 19 00:47:47 2022] GET /compute/v2.1/servers/85efeeae-0648-4a33-bc74-c0f0d25880aa => generated 1723 bytes in 301 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 19 00:47:48.372889 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-b324d9af-e99f-47a2-93d7-353cb956e835 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:47:48.373953 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-b324d9af-e99f-47a2-93d7-353cb956e835 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] [instance: b38d4219-8912-4fd9-9d02-7848016f7954] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:47:48.383359 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-b324d9af-e99f-47a2-93d7-353cb956e835 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:47:48.384345 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-b324d9af-e99f-47a2-93d7-353cb956e835 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:47:48.385032 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-b324d9af-e99f-47a2-93d7-353cb956e835 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:47:48.386316 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-b324d9af-e99f-47a2-93d7-353cb956e835 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:47:48.387226 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-b324d9af-e99f-47a2-93d7-353cb956e835 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:47:48.387845 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-b324d9af-e99f-47a2-93d7-353cb956e835 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:47:48.694757 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-b324d9af-e99f-47a2-93d7-353cb956e835 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a48ed2d9ded741ca91afc1fbbd9c608f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '82d7723ef95b4c3faeac9835eeb233a5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:47:48.706203 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-b324d9af-e99f-47a2-93d7-353cb956e835 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] 10.222.0.26 "GET /compute/v2.1/servers/b38d4219-8912-4fd9-9d02-7848016f7954" status: 200 len: 1715 microversion: 2.1 time: 0.339666 Oct 19 00:47:48.706984 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 698/813] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:47:48 2022] GET /compute/v2.1/servers/b38d4219-8912-4fd9-9d02-7848016f7954 => generated 1715 bytes in 342 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 19 00:47:48.840652 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-8ffa7257-bfa1-41e0-b5d7-4f77c2ce13e0 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:47:48.842536 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-8ffa7257-bfa1-41e0-b5d7-4f77c2ce13e0 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] [instance: 85efeeae-0648-4a33-bc74-c0f0d25880aa] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:47:48.857201 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-8ffa7257-bfa1-41e0-b5d7-4f77c2ce13e0 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:47:48.857442 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-8ffa7257-bfa1-41e0-b5d7-4f77c2ce13e0 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:47:48.858242 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-8ffa7257-bfa1-41e0-b5d7-4f77c2ce13e0 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:47:48.859650 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-8ffa7257-bfa1-41e0-b5d7-4f77c2ce13e0 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:47:48.860540 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-8ffa7257-bfa1-41e0-b5d7-4f77c2ce13e0 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:47:48.861359 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-8ffa7257-bfa1-41e0-b5d7-4f77c2ce13e0 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:47:49.184133 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-8ffa7257-bfa1-41e0-b5d7-4f77c2ce13e0 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:47:49.203157 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-8ffa7257-bfa1-41e0-b5d7-4f77c2ce13e0 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:47:49.209459 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-8ffa7257-bfa1-41e0-b5d7-4f77c2ce13e0 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:47:49.212814 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-8ffa7257-bfa1-41e0-b5d7-4f77c2ce13e0 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] 10.222.0.26 "GET /compute/v2.1/servers/85efeeae-0648-4a33-bc74-c0f0d25880aa" status: 200 len: 1723 microversion: 2.20 time: 0.379288 Oct 19 00:47:49.226521 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 699/814] 10.222.0.26 () {60 vars in 1277 bytes} [Wed Oct 19 00:47:48 2022] GET /compute/v2.1/servers/85efeeae-0648-4a33-bc74-c0f0d25880aa => generated 1723 bytes in 384 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 19 00:47:50.258156 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-b758e2a1-7c72-4ad7-9194-7a7511eefde3 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:47:50.259293 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-b758e2a1-7c72-4ad7-9194-7a7511eefde3 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] [instance: 85efeeae-0648-4a33-bc74-c0f0d25880aa] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:47:50.271087 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-b758e2a1-7c72-4ad7-9194-7a7511eefde3 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:47:50.271914 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-b758e2a1-7c72-4ad7-9194-7a7511eefde3 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:47:50.272657 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-b758e2a1-7c72-4ad7-9194-7a7511eefde3 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:47:50.273948 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-b758e2a1-7c72-4ad7-9194-7a7511eefde3 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:47:50.274745 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-b758e2a1-7c72-4ad7-9194-7a7511eefde3 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:47:50.275531 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-b758e2a1-7c72-4ad7-9194-7a7511eefde3 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:47:50.464820 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-b758e2a1-7c72-4ad7-9194-7a7511eefde3 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:47:50.476553 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-b758e2a1-7c72-4ad7-9194-7a7511eefde3 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:47:50.478394 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-b758e2a1-7c72-4ad7-9194-7a7511eefde3 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:47:50.479961 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-b758e2a1-7c72-4ad7-9194-7a7511eefde3 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] 10.222.0.26 "GET /compute/v2.1/servers/85efeeae-0648-4a33-bc74-c0f0d25880aa" status: 200 len: 1723 microversion: 2.20 time: 0.230173 Oct 19 00:47:50.480696 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 700/815] 10.222.0.26 () {60 vars in 1277 bytes} [Wed Oct 19 00:47:50 2022] GET /compute/v2.1/servers/85efeeae-0648-4a33-bc74-c0f0d25880aa => generated 1723 bytes in 232 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Oct 19 00:47:51.132837 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [req-85da3f56-2cb1-44e6-a43b-3c99609a3d72 req-138397d5-6f2d-4461-8fc8-072de561bed5 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "85efeeae-0648-4a33-bc74-c0f0d25880aa", "name": "network-vif-deleted", "tag": "ba933942-134c-497e-8475-ff284eee237e"}]} {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 19 00:47:51.152077 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-85da3f56-2cb1-44e6-a43b-3c99609a3d72 req-138397d5-6f2d-4461-8fc8-072de561bed5 service nova] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:47:51.161154 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-85da3f56-2cb1-44e6-a43b-3c99609a3d72 req-138397d5-6f2d-4461-8fc8-072de561bed5 service nova] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:47:51.163849 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-85da3f56-2cb1-44e6-a43b-3c99609a3d72 req-138397d5-6f2d-4461-8fc8-072de561bed5 service nova] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:47:51.203368 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.compute.server_external_events [req-85da3f56-2cb1-44e6-a43b-3c99609a3d72 req-138397d5-6f2d-4461-8fc8-072de561bed5 service nova] Creating event network-vif-deleted:ba933942-134c-497e-8475-ff284eee237e for instance 85efeeae-0648-4a33-bc74-c0f0d25880aa on hv-cinder-maste Oct 19 00:47:51.210985 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [req-85da3f56-2cb1-44e6-a43b-3c99609a3d72 req-138397d5-6f2d-4461-8fc8-072de561bed5 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.087297 Oct 19 00:47:51.212270 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 701/816] 10.222.0.26 () {64 vars in 1317 bytes} [Wed Oct 19 00:47:51 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 19 00:47:51.522109 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-84176b21-f328-479d-a758-73357b38edaf tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:47:51.525021 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-84176b21-f328-479d-a758-73357b38edaf tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] [instance: 85efeeae-0648-4a33-bc74-c0f0d25880aa] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:47:51.543162 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-84176b21-f328-479d-a758-73357b38edaf tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:47:51.544336 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-84176b21-f328-479d-a758-73357b38edaf tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:47:51.545394 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-84176b21-f328-479d-a758-73357b38edaf tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:47:51.547368 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-84176b21-f328-479d-a758-73357b38edaf tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:47:51.553381 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-84176b21-f328-479d-a758-73357b38edaf tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:47:51.558355 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-84176b21-f328-479d-a758-73357b38edaf tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:47:51.645162 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-84176b21-f328-479d-a758-73357b38edaf tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:47:51.645988 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-84176b21-f328-479d-a758-73357b38edaf tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:47:51.646756 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-84176b21-f328-479d-a758-73357b38edaf tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:47:51.647802 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.objects.instance [None req-84176b21-f328-479d-a758-73357b38edaf tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lazy-loading 'fault' on Instance uuid 85efeeae-0648-4a33-bc74-c0f0d25880aa {{(pid=125641) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 19 00:47:51.734547 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-84176b21-f328-479d-a758-73357b38edaf tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:47:51.752075 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-84176b21-f328-479d-a758-73357b38edaf tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:47:51.756126 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-84176b21-f328-479d-a758-73357b38edaf tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '69fef5650fff42e4ac4ed3ca74d4b123', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '37ffcffcae9b41bbb2d13adc49cb99df', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:47:51.758820 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-84176b21-f328-479d-a758-73357b38edaf tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] 10.222.0.26 "GET /compute/v2.1/servers/85efeeae-0648-4a33-bc74-c0f0d25880aa" status: 200 len: 1371 microversion: 2.20 time: 0.250500 Oct 19 00:47:51.759870 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 702/817] 10.222.0.26 () {60 vars in 1277 bytes} [Wed Oct 19 00:47:51 2022] GET /compute/v2.1/servers/85efeeae-0648-4a33-bc74-c0f0d25880aa => generated 1371 bytes in 253 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Oct 19 00:47:52.803369 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-554c199e-e667-404a-8e70-0bd4a65bdebc tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:47:52.804036 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-554c199e-e667-404a-8e70-0bd4a65bdebc tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] [instance: 85efeeae-0648-4a33-bc74-c0f0d25880aa] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:47:52.820104 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-554c199e-e667-404a-8e70-0bd4a65bdebc tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:47:52.820938 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-554c199e-e667-404a-8e70-0bd4a65bdebc tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:47:52.821593 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-554c199e-e667-404a-8e70-0bd4a65bdebc tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:47:52.822630 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-554c199e-e667-404a-8e70-0bd4a65bdebc tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:47:52.823332 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-554c199e-e667-404a-8e70-0bd4a65bdebc tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:47:52.823973 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-554c199e-e667-404a-8e70-0bd4a65bdebc tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:47:52.841588 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.wsgi [None req-554c199e-e667-404a-8e70-0bd4a65bdebc tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] HTTP exception thrown: Instance 85efeeae-0648-4a33-bc74-c0f0d25880aa could not be found. Oct 19 00:47:52.843562 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-554c199e-e667-404a-8e70-0bd4a65bdebc tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Request method failure captured: Oct 19 00:47:52.843562 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: request: GET /compute/v2.1/servers/85efeeae-0648-4a33-bc74-c0f0d25880aa HTTP/1.1 Oct 19 00:47:52.843562 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Accept: application/json Oct 19 00:47:52.843562 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Accept-Encoding: identity Oct 19 00:47:52.843562 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Connection: close Oct 19 00:47:52.843562 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Content-Length: 0 Oct 19 00:47:52.843562 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Content-Type: application/json Oct 19 00:47:52.843562 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Host: 10.222.0.26 Oct 19 00:47:52.843562 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Openstack-System-Scope: None Oct 19 00:47:52.843562 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: User-Agent: python-urllib3/1.26.12 Oct 19 00:47:52.843562 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Auth-Token: gAAAAABjT0eHghZZjQqIuLCd4UKjhyLHbqvZwQOzf8mMbPeH2urCSODAi7cb1nzpcISqPOiEw0ao4lxHIgJNa4VK2pIrrAVhMlfGsoDt_GEPXgOU1OcEl_JfKpXVIRpiEPvnqRQQAWlaMAyPJ6rWW3fskjR5vymFctg8L4gTCyMdOZ7pK8ZqEBg Oct 19 00:47:52.843562 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Domain-Id: None Oct 19 00:47:52.843562 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Domain-Name: None Oct 19 00:47:52.843562 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Identity-Status: Confirmed Oct 19 00:47:52.843562 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Is-Admin-Project: True Oct 19 00:47:52.843562 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Openstack-Nova-Api-Version: 2.20 Oct 19 00:47:52.843562 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Project-Domain-Id: default Oct 19 00:47:52.843562 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Project-Domain-Name: Default Oct 19 00:47:52.843562 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Project-Id: 37ffcffcae9b41bbb2d13adc49cb99df Oct 19 00:47:52.843562 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-495634389 Oct 19 00:47:52.843562 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Role: member,reader Oct 19 00:47:52.843562 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Roles: member,reader Oct 19 00:47:52.845307 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/37ffcffcae9b41bbb2d13adc49cb99df"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/37ffcffcae9b41bbb2d13adc49cb99df"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/37ffcffcae9b41bbb2d13adc49cb99df"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_37ffcffcae9b41bbb2d13adc49cb99df"}]}] Oct 19 00:47:52.845307 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Tenant: tempest-AttachVolumeShelveTestJSON-495634389 Oct 19 00:47:52.845307 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Tenant-Id: 37ffcffcae9b41bbb2d13adc49cb99df Oct 19 00:47:52.845307 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-495634389 Oct 19 00:47:52.845307 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User: tempest-AttachVolumeShelveTestJSON-495634389-project Oct 19 00:47:52.845307 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User-Domain-Id: default Oct 19 00:47:52.845307 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User-Domain-Name: Default Oct 19 00:47:52.845307 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User-Id: 69fef5650fff42e4ac4ed3ca74d4b123 Oct 19 00:47:52.845307 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User-Name: tempest-AttachVolumeShelveTestJSON-495634389-project Oct 19 00:47:52.845307 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: method: > Oct 19 00:47:52.845307 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: exception: Instance 85efeeae-0648-4a33-bc74-c0f0d25880aa could not be found. Oct 19 00:47:52.845307 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: action_args: {'id': '85efeeae-0648-4a33-bc74-c0f0d25880aa'} Oct 19 00:47:52.846102 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 19 00:47:52.846102 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 19 00:47:52.846102 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 19 00:47:52.846102 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 19 00:47:52.846102 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 19 00:47:52.846102 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 19 00:47:52.846102 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 19 00:47:52.846102 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 19 00:47:52.846102 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 19 00:47:52.846102 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 19 00:47:52.846102 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 85efeeae-0648-4a33-bc74-c0f0d25880aa could not be found. Oct 19 00:47:52.846102 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:47:52.846102 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 19 00:47:52.846102 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:47:52.846102 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 19 00:47:52.846102 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 19 00:47:52.846102 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 19 00:47:52.846102 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 19 00:47:52.846102 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 19 00:47:52.846102 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 19 00:47:52.848037 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 19 00:47:52.848037 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 19 00:47:52.848037 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 19 00:47:52.848037 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 19 00:47:52.848037 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 19 00:47:52.848037 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 19 00:47:52.848037 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 19 00:47:52.848037 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 85efeeae-0648-4a33-bc74-c0f0d25880aa could not be found. Oct 19 00:47:52.848037 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:47:52.848037 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 19 00:47:52.848037 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:47:52.848037 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 19 00:47:52.848037 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 19 00:47:52.848037 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 19 00:47:52.848037 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 19 00:47:52.848037 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 19 00:47:52.848037 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 85efeeae-0648-4a33-bc74-c0f0d25880aa could not be found. Oct 19 00:47:52.848037 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:47:52.851984 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-554c199e-e667-404a-8e70-0bd4a65bdebc tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Returning 404 to user: Instance 85efeeae-0648-4a33-bc74-c0f0d25880aa could not be found. {{(pid=125641) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 19 00:47:52.853403 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-554c199e-e667-404a-8e70-0bd4a65bdebc tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] 10.222.0.26 "GET /compute/v2.1/servers/85efeeae-0648-4a33-bc74-c0f0d25880aa" status: 404 len: 111 microversion: 2.20 time: 0.054888 Oct 19 00:47:52.854065 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 703/818] 10.222.0.26 () {60 vars in 1277 bytes} [Wed Oct 19 00:47:52 2022] GET /compute/v2.1/servers/85efeeae-0648-4a33-bc74-c0f0d25880aa => generated 111 bytes in 56 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Oct 19 00:47:52.871764 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-f5c0be63-695f-4a03-83fa-aec6a8054dd2 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Calling method '.version_select at 0x7f71cb5cf9d0>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:47:52.895571 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-f5c0be63-695f-4a03-83fa-aec6a8054dd2 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] 10.222.0.26 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-936267011" status: 204 len: 0 microversion: 2.20 time: 0.027724 Oct 19 00:47:52.896256 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 704/819] 10.222.0.26 () {60 vars in 1259 bytes} [Wed Oct 19 00:47:52 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-936267011 => generated 0 bytes in 29 msecs (HTTP/1.1 204) 9 headers in 365 bytes (1 switches on core 0) Oct 19 00:47:55.038040 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-16063f60-8c7a-40fe-ae8c-8fe99f8437a6 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:47:55.038850 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-16063f60-8c7a-40fe-ae8c-8fe99f8437a6 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] [instance: 85efeeae-0648-4a33-bc74-c0f0d25880aa] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:47:55.046304 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-16063f60-8c7a-40fe-ae8c-8fe99f8437a6 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:47:55.046824 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-16063f60-8c7a-40fe-ae8c-8fe99f8437a6 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:47:55.047265 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-16063f60-8c7a-40fe-ae8c-8fe99f8437a6 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:47:55.047980 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-16063f60-8c7a-40fe-ae8c-8fe99f8437a6 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:47:55.048430 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-16063f60-8c7a-40fe-ae8c-8fe99f8437a6 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:47:55.048854 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-16063f60-8c7a-40fe-ae8c-8fe99f8437a6 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:47:55.064058 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.wsgi [None req-16063f60-8c7a-40fe-ae8c-8fe99f8437a6 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] HTTP exception thrown: Instance 85efeeae-0648-4a33-bc74-c0f0d25880aa could not be found. Oct 19 00:47:55.065691 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-16063f60-8c7a-40fe-ae8c-8fe99f8437a6 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Request method failure captured: Oct 19 00:47:55.065691 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: request: DELETE /compute/v2.1/servers/85efeeae-0648-4a33-bc74-c0f0d25880aa HTTP/1.1 Oct 19 00:47:55.065691 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Accept: application/json Oct 19 00:47:55.065691 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Accept-Encoding: identity Oct 19 00:47:55.065691 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Connection: close Oct 19 00:47:55.065691 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Content-Length: 0 Oct 19 00:47:55.065691 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Content-Type: application/json Oct 19 00:47:55.065691 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Host: 10.222.0.26 Oct 19 00:47:55.065691 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Openstack-System-Scope: None Oct 19 00:47:55.065691 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: User-Agent: python-urllib3/1.26.12 Oct 19 00:47:55.065691 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Auth-Token: gAAAAABjT0eHghZZjQqIuLCd4UKjhyLHbqvZwQOzf8mMbPeH2urCSODAi7cb1nzpcISqPOiEw0ao4lxHIgJNa4VK2pIrrAVhMlfGsoDt_GEPXgOU1OcEl_JfKpXVIRpiEPvnqRQQAWlaMAyPJ6rWW3fskjR5vymFctg8L4gTCyMdOZ7pK8ZqEBg Oct 19 00:47:55.065691 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Domain-Id: None Oct 19 00:47:55.065691 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Domain-Name: None Oct 19 00:47:55.065691 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Identity-Status: Confirmed Oct 19 00:47:55.065691 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Is-Admin-Project: True Oct 19 00:47:55.065691 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Openstack-Nova-Api-Version: 2.20 Oct 19 00:47:55.065691 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Project-Domain-Id: default Oct 19 00:47:55.065691 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Project-Domain-Name: Default Oct 19 00:47:55.065691 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Project-Id: 37ffcffcae9b41bbb2d13adc49cb99df Oct 19 00:47:55.065691 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-495634389 Oct 19 00:47:55.065691 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Role: member,reader Oct 19 00:47:55.065691 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Roles: member,reader Oct 19 00:47:55.068335 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/37ffcffcae9b41bbb2d13adc49cb99df"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/37ffcffcae9b41bbb2d13adc49cb99df"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/37ffcffcae9b41bbb2d13adc49cb99df"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_37ffcffcae9b41bbb2d13adc49cb99df"}]}] Oct 19 00:47:55.068335 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Tenant: tempest-AttachVolumeShelveTestJSON-495634389 Oct 19 00:47:55.068335 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Tenant-Id: 37ffcffcae9b41bbb2d13adc49cb99df Oct 19 00:47:55.068335 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-495634389 Oct 19 00:47:55.068335 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User: tempest-AttachVolumeShelveTestJSON-495634389-project Oct 19 00:47:55.068335 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User-Domain-Id: default Oct 19 00:47:55.068335 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User-Domain-Name: Default Oct 19 00:47:55.068335 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User-Id: 69fef5650fff42e4ac4ed3ca74d4b123 Oct 19 00:47:55.068335 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User-Name: tempest-AttachVolumeShelveTestJSON-495634389-project Oct 19 00:47:55.068335 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: method: > Oct 19 00:47:55.068335 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: exception: Instance 85efeeae-0648-4a33-bc74-c0f0d25880aa could not be found. Oct 19 00:47:55.068335 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: action_args: {'id': '85efeeae-0648-4a33-bc74-c0f0d25880aa'} Oct 19 00:47:55.069882 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 19 00:47:55.069882 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 19 00:47:55.069882 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 19 00:47:55.069882 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 19 00:47:55.069882 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 19 00:47:55.069882 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 19 00:47:55.069882 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 19 00:47:55.069882 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 19 00:47:55.069882 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 19 00:47:55.069882 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 19 00:47:55.069882 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 85efeeae-0648-4a33-bc74-c0f0d25880aa could not be found. Oct 19 00:47:55.069882 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:47:55.069882 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 19 00:47:55.069882 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:47:55.069882 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 19 00:47:55.069882 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 19 00:47:55.069882 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 19 00:47:55.069882 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 19 00:47:55.069882 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 19 00:47:55.069882 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 19 00:47:55.072287 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 19 00:47:55.072287 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Oct 19 00:47:55.072287 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Oct 19 00:47:55.072287 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Oct 19 00:47:55.072287 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Oct 19 00:47:55.072287 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 19 00:47:55.072287 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 19 00:47:55.072287 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 19 00:47:55.072287 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 19 00:47:55.072287 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 85efeeae-0648-4a33-bc74-c0f0d25880aa could not be found. Oct 19 00:47:55.072287 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:47:55.072287 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 19 00:47:55.072287 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:47:55.072287 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 19 00:47:55.072287 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 19 00:47:55.072287 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 19 00:47:55.072287 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 19 00:47:55.072287 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 19 00:47:55.072287 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 85efeeae-0648-4a33-bc74-c0f0d25880aa could not be found. Oct 19 00:47:55.072287 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:47:55.074804 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-16063f60-8c7a-40fe-ae8c-8fe99f8437a6 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Returning 404 to user: Instance 85efeeae-0648-4a33-bc74-c0f0d25880aa could not be found. {{(pid=125641) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 19 00:47:55.074804 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-16063f60-8c7a-40fe-ae8c-8fe99f8437a6 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] 10.222.0.26 "DELETE /compute/v2.1/servers/85efeeae-0648-4a33-bc74-c0f0d25880aa" status: 404 len: 111 microversion: 2.20 time: 0.034536 Oct 19 00:47:55.074804 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 705/820] 10.222.0.26 () {60 vars in 1280 bytes} [Wed Oct 19 00:47:55 2022] DELETE /compute/v2.1/servers/85efeeae-0648-4a33-bc74-c0f0d25880aa => generated 111 bytes in 36 msecs (HTTP/1.1 404) 9 headers in 381 bytes (2 switches on core 0) Oct 19 00:47:55.076121 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-33117a49-d084-4c53-9048-e370fc4c4014 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:47:55.076739 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-33117a49-d084-4c53-9048-e370fc4c4014 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] [instance: 85efeeae-0648-4a33-bc74-c0f0d25880aa] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:47:55.083447 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-33117a49-d084-4c53-9048-e370fc4c4014 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:47:55.083937 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-33117a49-d084-4c53-9048-e370fc4c4014 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:47:55.084374 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-33117a49-d084-4c53-9048-e370fc4c4014 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:47:55.085083 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-33117a49-d084-4c53-9048-e370fc4c4014 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:47:55.085540 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-33117a49-d084-4c53-9048-e370fc4c4014 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:47:55.085958 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-33117a49-d084-4c53-9048-e370fc4c4014 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:47:55.102082 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.wsgi [None req-33117a49-d084-4c53-9048-e370fc4c4014 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] HTTP exception thrown: Instance 85efeeae-0648-4a33-bc74-c0f0d25880aa could not be found. Oct 19 00:47:55.103671 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-33117a49-d084-4c53-9048-e370fc4c4014 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Request method failure captured: Oct 19 00:47:55.103671 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: request: GET /compute/v2.1/servers/85efeeae-0648-4a33-bc74-c0f0d25880aa HTTP/1.1 Oct 19 00:47:55.103671 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Accept: application/json Oct 19 00:47:55.103671 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Accept-Encoding: identity Oct 19 00:47:55.103671 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Connection: close Oct 19 00:47:55.103671 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Content-Length: 0 Oct 19 00:47:55.103671 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Content-Type: application/json Oct 19 00:47:55.103671 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Host: 10.222.0.26 Oct 19 00:47:55.103671 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Openstack-System-Scope: None Oct 19 00:47:55.103671 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: User-Agent: python-urllib3/1.26.12 Oct 19 00:47:55.103671 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Auth-Token: gAAAAABjT0eHghZZjQqIuLCd4UKjhyLHbqvZwQOzf8mMbPeH2urCSODAi7cb1nzpcISqPOiEw0ao4lxHIgJNa4VK2pIrrAVhMlfGsoDt_GEPXgOU1OcEl_JfKpXVIRpiEPvnqRQQAWlaMAyPJ6rWW3fskjR5vymFctg8L4gTCyMdOZ7pK8ZqEBg Oct 19 00:47:55.103671 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Domain-Id: None Oct 19 00:47:55.103671 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Domain-Name: None Oct 19 00:47:55.103671 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Identity-Status: Confirmed Oct 19 00:47:55.103671 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Is-Admin-Project: True Oct 19 00:47:55.103671 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Openstack-Nova-Api-Version: 2.20 Oct 19 00:47:55.103671 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Project-Domain-Id: default Oct 19 00:47:55.103671 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Project-Domain-Name: Default Oct 19 00:47:55.103671 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Project-Id: 37ffcffcae9b41bbb2d13adc49cb99df Oct 19 00:47:55.103671 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-495634389 Oct 19 00:47:55.103671 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Role: member,reader Oct 19 00:47:55.103671 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Roles: member,reader Oct 19 00:47:55.106215 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/37ffcffcae9b41bbb2d13adc49cb99df"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/37ffcffcae9b41bbb2d13adc49cb99df"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/37ffcffcae9b41bbb2d13adc49cb99df"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_37ffcffcae9b41bbb2d13adc49cb99df"}]}] Oct 19 00:47:55.106215 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Tenant: tempest-AttachVolumeShelveTestJSON-495634389 Oct 19 00:47:55.106215 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Tenant-Id: 37ffcffcae9b41bbb2d13adc49cb99df Oct 19 00:47:55.106215 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-495634389 Oct 19 00:47:55.106215 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User: tempest-AttachVolumeShelveTestJSON-495634389-project Oct 19 00:47:55.106215 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User-Domain-Id: default Oct 19 00:47:55.106215 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User-Domain-Name: Default Oct 19 00:47:55.106215 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User-Id: 69fef5650fff42e4ac4ed3ca74d4b123 Oct 19 00:47:55.106215 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User-Name: tempest-AttachVolumeShelveTestJSON-495634389-project Oct 19 00:47:55.106215 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: method: > Oct 19 00:47:55.106215 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: exception: Instance 85efeeae-0648-4a33-bc74-c0f0d25880aa could not be found. Oct 19 00:47:55.106215 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: action_args: {'id': '85efeeae-0648-4a33-bc74-c0f0d25880aa'} Oct 19 00:47:55.107958 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 19 00:47:55.107958 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 19 00:47:55.107958 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 19 00:47:55.107958 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 19 00:47:55.107958 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 19 00:47:55.107958 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 19 00:47:55.107958 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 19 00:47:55.107958 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 19 00:47:55.107958 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 19 00:47:55.107958 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 19 00:47:55.107958 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 85efeeae-0648-4a33-bc74-c0f0d25880aa could not be found. Oct 19 00:47:55.107958 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:47:55.107958 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 19 00:47:55.107958 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:47:55.107958 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 19 00:47:55.107958 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 19 00:47:55.107958 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 19 00:47:55.107958 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 19 00:47:55.107958 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 19 00:47:55.107958 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 19 00:47:55.110395 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 19 00:47:55.110395 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 19 00:47:55.110395 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 19 00:47:55.110395 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 19 00:47:55.110395 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 19 00:47:55.110395 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 19 00:47:55.110395 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 19 00:47:55.110395 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 85efeeae-0648-4a33-bc74-c0f0d25880aa could not be found. Oct 19 00:47:55.110395 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:47:55.110395 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 19 00:47:55.110395 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:47:55.110395 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 19 00:47:55.110395 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 19 00:47:55.110395 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 19 00:47:55.110395 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 19 00:47:55.110395 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 19 00:47:55.110395 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 85efeeae-0648-4a33-bc74-c0f0d25880aa could not be found. Oct 19 00:47:55.110395 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:47:55.113227 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-33117a49-d084-4c53-9048-e370fc4c4014 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Returning 404 to user: Instance 85efeeae-0648-4a33-bc74-c0f0d25880aa could not be found. {{(pid=125641) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 19 00:47:55.113227 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-33117a49-d084-4c53-9048-e370fc4c4014 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] 10.222.0.26 "GET /compute/v2.1/servers/85efeeae-0648-4a33-bc74-c0f0d25880aa" status: 404 len: 111 microversion: 2.20 time: 0.033015 Oct 19 00:47:55.113227 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 706/821] 10.222.0.26 () {60 vars in 1277 bytes} [Wed Oct 19 00:47:55 2022] GET /compute/v2.1/servers/85efeeae-0648-4a33-bc74-c0f0d25880aa => generated 111 bytes in 34 msecs (HTTP/1.1 404) 9 headers in 381 bytes (2 switches on core 0) Oct 19 00:47:55.433211 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-16e621e4-10ba-4c14-bf18-8dd165138e97 tempest-VolumesTestJSON-1337473070 tempest-VolumesTestJSON-1337473070-project] Calling method '.version_select at 0x7f71c820f8b0>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:47:55.496069 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-16e621e4-10ba-4c14-bf18-8dd165138e97 tempest-VolumesTestJSON-1337473070 tempest-VolumesTestJSON-1337473070-project] 10.222.0.26 "GET /compute/v2.1/os-volumes/9178777a-0173-436e-8915-1004e617ae2f" status: 200 len: 350 microversion: 2.1 time: 0.066656 Oct 19 00:47:55.496786 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 707/822] 10.222.0.26 () {58 vars in 1245 bytes} [Wed Oct 19 00:47:55 2022] GET /compute/v2.1/os-volumes/9178777a-0173-436e-8915-1004e617ae2f => generated 350 bytes in 68 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 19 00:47:56.290993 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-951b8f82-8fd5-4163-8352-08b868cf3749 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:47:56.291858 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-951b8f82-8fd5-4163-8352-08b868cf3749 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] [instance: b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:47:56.300229 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-951b8f82-8fd5-4163-8352-08b868cf3749 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:47:56.301187 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-951b8f82-8fd5-4163-8352-08b868cf3749 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:47:56.301372 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-951b8f82-8fd5-4163-8352-08b868cf3749 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:47:56.302136 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-951b8f82-8fd5-4163-8352-08b868cf3749 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:47:56.302771 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-951b8f82-8fd5-4163-8352-08b868cf3749 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:47:56.303285 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-951b8f82-8fd5-4163-8352-08b868cf3749 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:47:56.319286 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.wsgi [None req-951b8f82-8fd5-4163-8352-08b868cf3749 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] HTTP exception thrown: Instance b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2 could not be found. Oct 19 00:47:56.320892 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-951b8f82-8fd5-4163-8352-08b868cf3749 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Request method failure captured: Oct 19 00:47:56.320892 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: request: DELETE /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2 HTTP/1.1 Oct 19 00:47:56.320892 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Accept: application/json Oct 19 00:47:56.320892 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Accept-Encoding: identity Oct 19 00:47:56.320892 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Connection: close Oct 19 00:47:56.320892 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Content-Length: 0 Oct 19 00:47:56.320892 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Content-Type: application/json Oct 19 00:47:56.320892 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Host: 10.222.0.26 Oct 19 00:47:56.320892 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Openstack-System-Scope: None Oct 19 00:47:56.320892 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: User-Agent: python-urllib3/1.26.12 Oct 19 00:47:56.320892 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Auth-Token: gAAAAABjT0eHghZZjQqIuLCd4UKjhyLHbqvZwQOzf8mMbPeH2urCSODAi7cb1nzpcISqPOiEw0ao4lxHIgJNa4VK2pIrrAVhMlfGsoDt_GEPXgOU1OcEl_JfKpXVIRpiEPvnqRQQAWlaMAyPJ6rWW3fskjR5vymFctg8L4gTCyMdOZ7pK8ZqEBg Oct 19 00:47:56.320892 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Domain-Id: None Oct 19 00:47:56.320892 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Domain-Name: None Oct 19 00:47:56.320892 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Identity-Status: Confirmed Oct 19 00:47:56.320892 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Is-Admin-Project: True Oct 19 00:47:56.320892 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Openstack-Nova-Api-Version: 2.20 Oct 19 00:47:56.320892 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Project-Domain-Id: default Oct 19 00:47:56.320892 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Project-Domain-Name: Default Oct 19 00:47:56.320892 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Project-Id: 37ffcffcae9b41bbb2d13adc49cb99df Oct 19 00:47:56.320892 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-495634389 Oct 19 00:47:56.320892 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Role: member,reader Oct 19 00:47:56.320892 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Roles: member,reader Oct 19 00:47:56.323652 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/37ffcffcae9b41bbb2d13adc49cb99df"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/37ffcffcae9b41bbb2d13adc49cb99df"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/37ffcffcae9b41bbb2d13adc49cb99df"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_37ffcffcae9b41bbb2d13adc49cb99df"}]}] Oct 19 00:47:56.323652 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Tenant: tempest-AttachVolumeShelveTestJSON-495634389 Oct 19 00:47:56.323652 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Tenant-Id: 37ffcffcae9b41bbb2d13adc49cb99df Oct 19 00:47:56.323652 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-495634389 Oct 19 00:47:56.323652 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User: tempest-AttachVolumeShelveTestJSON-495634389-project Oct 19 00:47:56.323652 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User-Domain-Id: default Oct 19 00:47:56.323652 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User-Domain-Name: Default Oct 19 00:47:56.323652 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User-Id: 69fef5650fff42e4ac4ed3ca74d4b123 Oct 19 00:47:56.323652 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User-Name: tempest-AttachVolumeShelveTestJSON-495634389-project Oct 19 00:47:56.323652 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: method: > Oct 19 00:47:56.323652 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: exception: Instance b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2 could not be found. Oct 19 00:47:56.323652 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: action_args: {'id': 'b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2'} Oct 19 00:47:56.325206 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 19 00:47:56.325206 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 19 00:47:56.325206 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 19 00:47:56.325206 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 19 00:47:56.325206 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 19 00:47:56.325206 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 19 00:47:56.325206 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 19 00:47:56.325206 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 19 00:47:56.325206 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 19 00:47:56.325206 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 19 00:47:56.325206 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2 could not be found. Oct 19 00:47:56.325206 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:47:56.325206 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 19 00:47:56.325206 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:47:56.325206 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 19 00:47:56.325206 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 19 00:47:56.325206 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 19 00:47:56.325206 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 19 00:47:56.325206 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 19 00:47:56.325206 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 19 00:47:56.328036 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 19 00:47:56.328036 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Oct 19 00:47:56.328036 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Oct 19 00:47:56.328036 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Oct 19 00:47:56.328036 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Oct 19 00:47:56.328036 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 19 00:47:56.328036 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 19 00:47:56.328036 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 19 00:47:56.328036 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 19 00:47:56.328036 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2 could not be found. Oct 19 00:47:56.328036 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:47:56.328036 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 19 00:47:56.328036 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:47:56.328036 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 19 00:47:56.328036 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 19 00:47:56.328036 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 19 00:47:56.328036 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 19 00:47:56.328036 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 19 00:47:56.328036 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2 could not be found. Oct 19 00:47:56.328036 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:47:56.330800 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-951b8f82-8fd5-4163-8352-08b868cf3749 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Returning 404 to user: Instance b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2 could not be found. {{(pid=125641) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 19 00:47:56.330800 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-951b8f82-8fd5-4163-8352-08b868cf3749 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] 10.222.0.26 "DELETE /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2" status: 404 len: 111 microversion: 2.20 time: 0.037699 Oct 19 00:47:56.330800 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 708/823] 10.222.0.26 () {60 vars in 1280 bytes} [Wed Oct 19 00:47:56 2022] DELETE /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2 => generated 111 bytes in 40 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Oct 19 00:47:56.336523 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-f7bb4909-bf6b-480d-90f1-5cd908ad4315 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:47:56.337294 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-f7bb4909-bf6b-480d-90f1-5cd908ad4315 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] [instance: b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:47:56.344452 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-f7bb4909-bf6b-480d-90f1-5cd908ad4315 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:47:56.345007 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-f7bb4909-bf6b-480d-90f1-5cd908ad4315 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:47:56.345514 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-f7bb4909-bf6b-480d-90f1-5cd908ad4315 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:47:56.346374 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-f7bb4909-bf6b-480d-90f1-5cd908ad4315 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:47:56.347014 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-f7bb4909-bf6b-480d-90f1-5cd908ad4315 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:47:56.347637 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-f7bb4909-bf6b-480d-90f1-5cd908ad4315 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:47:56.367569 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.wsgi [None req-f7bb4909-bf6b-480d-90f1-5cd908ad4315 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] HTTP exception thrown: Instance b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2 could not be found. Oct 19 00:47:56.369301 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-f7bb4909-bf6b-480d-90f1-5cd908ad4315 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Request method failure captured: Oct 19 00:47:56.369301 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: request: GET /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2 HTTP/1.1 Oct 19 00:47:56.369301 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Accept: application/json Oct 19 00:47:56.369301 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Accept-Encoding: identity Oct 19 00:47:56.369301 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Connection: close Oct 19 00:47:56.369301 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Content-Length: 0 Oct 19 00:47:56.369301 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Content-Type: application/json Oct 19 00:47:56.369301 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Host: 10.222.0.26 Oct 19 00:47:56.369301 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Openstack-System-Scope: None Oct 19 00:47:56.369301 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: User-Agent: python-urllib3/1.26.12 Oct 19 00:47:56.369301 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Auth-Token: gAAAAABjT0eHghZZjQqIuLCd4UKjhyLHbqvZwQOzf8mMbPeH2urCSODAi7cb1nzpcISqPOiEw0ao4lxHIgJNa4VK2pIrrAVhMlfGsoDt_GEPXgOU1OcEl_JfKpXVIRpiEPvnqRQQAWlaMAyPJ6rWW3fskjR5vymFctg8L4gTCyMdOZ7pK8ZqEBg Oct 19 00:47:56.369301 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Domain-Id: None Oct 19 00:47:56.369301 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Domain-Name: None Oct 19 00:47:56.369301 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Identity-Status: Confirmed Oct 19 00:47:56.369301 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Is-Admin-Project: True Oct 19 00:47:56.369301 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Openstack-Nova-Api-Version: 2.20 Oct 19 00:47:56.369301 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Project-Domain-Id: default Oct 19 00:47:56.369301 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Project-Domain-Name: Default Oct 19 00:47:56.369301 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Project-Id: 37ffcffcae9b41bbb2d13adc49cb99df Oct 19 00:47:56.369301 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-495634389 Oct 19 00:47:56.369301 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Role: member,reader Oct 19 00:47:56.369301 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Roles: member,reader Oct 19 00:47:56.371951 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/37ffcffcae9b41bbb2d13adc49cb99df"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/37ffcffcae9b41bbb2d13adc49cb99df"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/37ffcffcae9b41bbb2d13adc49cb99df"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_37ffcffcae9b41bbb2d13adc49cb99df"}]}] Oct 19 00:47:56.371951 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Tenant: tempest-AttachVolumeShelveTestJSON-495634389 Oct 19 00:47:56.371951 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Tenant-Id: 37ffcffcae9b41bbb2d13adc49cb99df Oct 19 00:47:56.371951 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-495634389 Oct 19 00:47:56.371951 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User: tempest-AttachVolumeShelveTestJSON-495634389-project Oct 19 00:47:56.371951 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User-Domain-Id: default Oct 19 00:47:56.371951 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User-Domain-Name: Default Oct 19 00:47:56.371951 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User-Id: 69fef5650fff42e4ac4ed3ca74d4b123 Oct 19 00:47:56.371951 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User-Name: tempest-AttachVolumeShelveTestJSON-495634389-project Oct 19 00:47:56.371951 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: method: > Oct 19 00:47:56.371951 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: exception: Instance b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2 could not be found. Oct 19 00:47:56.371951 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: action_args: {'id': 'b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2'} Oct 19 00:47:56.373332 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 19 00:47:56.373332 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 19 00:47:56.373332 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 19 00:47:56.373332 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 19 00:47:56.373332 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 19 00:47:56.373332 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 19 00:47:56.373332 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 19 00:47:56.373332 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 19 00:47:56.373332 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 19 00:47:56.373332 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 19 00:47:56.373332 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2 could not be found. Oct 19 00:47:56.373332 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:47:56.373332 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 19 00:47:56.373332 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:47:56.373332 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 19 00:47:56.373332 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 19 00:47:56.373332 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 19 00:47:56.373332 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 19 00:47:56.373332 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 19 00:47:56.373332 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 19 00:47:56.375977 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 19 00:47:56.375977 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 19 00:47:56.375977 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 19 00:47:56.375977 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 19 00:47:56.375977 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 19 00:47:56.375977 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 19 00:47:56.375977 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 19 00:47:56.375977 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2 could not be found. Oct 19 00:47:56.375977 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:47:56.375977 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 19 00:47:56.375977 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:47:56.375977 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 19 00:47:56.375977 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 19 00:47:56.375977 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 19 00:47:56.375977 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 19 00:47:56.375977 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 19 00:47:56.375977 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2 could not be found. Oct 19 00:47:56.375977 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:47:56.379505 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-f7bb4909-bf6b-480d-90f1-5cd908ad4315 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] Returning 404 to user: Instance b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2 could not be found. {{(pid=125641) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 19 00:47:56.379505 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-f7bb4909-bf6b-480d-90f1-5cd908ad4315 tempest-AttachVolumeShelveTestJSON-495634389 tempest-AttachVolumeShelveTestJSON-495634389-project] 10.222.0.26 "GET /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2" status: 404 len: 111 microversion: 2.20 time: 0.039972 Oct 19 00:47:56.379505 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 709/824] 10.222.0.26 () {60 vars in 1277 bytes} [Wed Oct 19 00:47:56 2022] GET /compute/v2.1/servers/b2dadc2d-e0a5-4687-a04e-c78f2a17f8b2 => generated 111 bytes in 42 msecs (HTTP/1.1 404) 9 headers in 381 bytes (2 switches on core 0) Oct 19 00:47:57.050379 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-eb35b1b0-02ca-4f98-9c3b-9f5df0a8a23a tempest-VolumesTestJSON-1337473070 tempest-VolumesTestJSON-1337473070-project] Calling method '.version_select at 0x7f71c3f78d30>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:47:57.135942 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-eb35b1b0-02ca-4f98-9c3b-9f5df0a8a23a tempest-VolumesTestJSON-1337473070 tempest-VolumesTestJSON-1337473070-project] 10.222.0.26 "GET /compute/v2.1/os-volumes/06440412-b33b-4c0d-bd17-52cef7464a0f" status: 200 len: 350 microversion: 2.1 time: 0.090057 Oct 19 00:47:57.136872 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 710/825] 10.222.0.26 () {58 vars in 1245 bytes} [Wed Oct 19 00:47:57 2022] GET /compute/v2.1/os-volumes/06440412-b33b-4c0d-bd17-52cef7464a0f => generated 350 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 19 00:47:58.703816 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-754cfbd5-0629-4b16-9b95-50cccd70b4be tempest-VolumesTestJSON-1337473070 tempest-VolumesTestJSON-1337473070-project] Calling method '.version_select at 0x7f71cb5cf670>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:47:58.806791 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-754cfbd5-0629-4b16-9b95-50cccd70b4be tempest-VolumesTestJSON-1337473070 tempest-VolumesTestJSON-1337473070-project] 10.222.0.26 "GET /compute/v2.1/os-volumes/860aa132-b0a5-4ef0-9f24-3b153a2dde20" status: 200 len: 350 microversion: 2.1 time: 0.106825 Oct 19 00:47:58.807476 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 711/826] 10.222.0.26 () {58 vars in 1245 bytes} [Wed Oct 19 00:47:58 2022] GET /compute/v2.1/os-volumes/860aa132-b0a5-4ef0-9f24-3b153a2dde20 => generated 350 bytes in 109 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 19 00:47:58.822902 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-6ecb780f-953e-4a89-8378-6a48e2f316f6 tempest-VolumesTestJSON-1337473070 tempest-VolumesTestJSON-1337473070-project] Calling method '.version_select at 0x7f71cb5d4040>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:47:58.932156 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-6ecb780f-953e-4a89-8378-6a48e2f316f6 tempest-VolumesTestJSON-1337473070 tempest-VolumesTestJSON-1337473070-project] 10.222.0.26 "GET /compute/v2.1/os-volumes" status: 200 len: 1033 microversion: 2.1 time: 0.113391 Oct 19 00:47:58.943514 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 712/827] 10.222.0.26 () {58 vars in 1134 bytes} [Wed Oct 19 00:47:58 2022] GET /compute/v2.1/os-volumes => generated 1033 bytes in 125 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:47:58.951255 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-1ba99c5a-b35b-4e45-8fa7-375731c2fb75 tempest-VolumesTestJSON-1337473070 tempest-VolumesTestJSON-1337473070-project] Calling method '.version_select at 0x7f71cb5cfb80>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:47:59.098977 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-1ba99c5a-b35b-4e45-8fa7-375731c2fb75 tempest-VolumesTestJSON-1337473070 tempest-VolumesTestJSON-1337473070-project] 10.222.0.26 "GET /compute/v2.1/os-volumes?limit=2" status: 200 len: 693 microversion: 2.1 time: 0.152546 Oct 19 00:47:59.116196 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 713/828] 10.222.0.26 () {58 vars in 1149 bytes} [Wed Oct 19 00:47:58 2022] GET /compute/v2.1/os-volumes?limit=2 => generated 693 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 19 00:47:59.128053 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-9e4de005-b921-4347-90e2-1270413bfe9d tempest-VolumesTestJSON-1337473070 tempest-VolumesTestJSON-1337473070-project] Calling method '.version_select at 0x7f71c3ec0f70>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:47:59.327744 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-9e4de005-b921-4347-90e2-1270413bfe9d tempest-VolumesTestJSON-1337473070 tempest-VolumesTestJSON-1337473070-project] 10.222.0.26 "GET /compute/v2.1/os-volumes" status: 200 len: 1033 microversion: 2.1 time: 0.207730 Oct 19 00:47:59.343669 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 714/829] 10.222.0.26 () {58 vars in 1134 bytes} [Wed Oct 19 00:47:59 2022] GET /compute/v2.1/os-volumes => generated 1033 bytes in 225 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 19 00:47:59.358699 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-15582ed2-6a3f-4e40-bda0-828f6ab60ee8 tempest-VolumesTestJSON-1337473070 tempest-VolumesTestJSON-1337473070-project] Calling method '.version_select at 0x7f71cb5394c0>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:47:59.523209 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-15582ed2-6a3f-4e40-bda0-828f6ab60ee8 tempest-VolumesTestJSON-1337473070 tempest-VolumesTestJSON-1337473070-project] 10.222.0.26 "GET /compute/v2.1/os-volumes?offset=1&limit=1" status: 200 len: 353 microversion: 2.1 time: 0.172105 Oct 19 00:47:59.524471 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 715/830] 10.222.0.26 () {58 vars in 1167 bytes} [Wed Oct 19 00:47:59 2022] GET /compute/v2.1/os-volumes?offset=1&limit=1 => generated 353 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 19 00:47:59.553889 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-faec0bea-3c46-4af0-b9ba-8d186334471c tempest-VolumesTestJSON-1337473070 tempest-VolumesTestJSON-1337473070-project] Calling method '.version_select at 0x7f71c81f4940>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:47:59.661734 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-faec0bea-3c46-4af0-b9ba-8d186334471c tempest-VolumesTestJSON-1337473070 tempest-VolumesTestJSON-1337473070-project] 10.222.0.26 "GET /compute/v2.1/os-volumes/detail?limit=2" status: 200 len: 693 microversion: 2.1 time: 0.115079 Oct 19 00:47:59.663435 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 716/831] 10.222.0.26 () {58 vars in 1170 bytes} [Wed Oct 19 00:47:59 2022] GET /compute/v2.1/os-volumes/detail?limit=2 => generated 693 bytes in 118 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 19 00:47:59.691574 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-f294f46f-2d84-44c6-8e77-199bcde62847 tempest-VolumesTestJSON-1337473070 tempest-VolumesTestJSON-1337473070-project] Calling method '.version_select at 0x7f71c3f25e50>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:47:59.887813 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-f294f46f-2d84-44c6-8e77-199bcde62847 tempest-VolumesTestJSON-1337473070 tempest-VolumesTestJSON-1337473070-project] 10.222.0.26 "GET /compute/v2.1/os-volumes/detail" status: 200 len: 1033 microversion: 2.1 time: 0.204695 Oct 19 00:47:59.888956 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 717/832] 10.222.0.26 () {58 vars in 1155 bytes} [Wed Oct 19 00:47:59 2022] GET /compute/v2.1/os-volumes/detail => generated 1033 bytes in 207 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 19 00:47:59.904807 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-0f5aedeb-b928-4009-b87c-00598fc3484a tempest-VolumesTestJSON-1337473070 tempest-VolumesTestJSON-1337473070-project] Calling method '.version_select at 0x7f71c3ff1670>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:48:00.211825 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-0f5aedeb-b928-4009-b87c-00598fc3484a tempest-VolumesTestJSON-1337473070 tempest-VolumesTestJSON-1337473070-project] 10.222.0.26 "GET /compute/v2.1/os-volumes/detail?offset=1&limit=1" status: 200 len: 353 microversion: 2.1 time: 0.310588 Oct 19 00:48:00.223039 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 718/833] 10.222.0.26 () {58 vars in 1188 bytes} [Wed Oct 19 00:47:59 2022] GET /compute/v2.1/os-volumes/detail?offset=1&limit=1 => generated 353 bytes in 319 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 19 00:48:00.237996 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-214a7d2c-47bc-4b54-b3ba-4c542b11dfc4 tempest-VolumesTestJSON-1337473070 tempest-VolumesTestJSON-1337473070-project] Calling method '.version_select at 0x7f71cb5d4dc0>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:48:00.519718 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-214a7d2c-47bc-4b54-b3ba-4c542b11dfc4 tempest-VolumesTestJSON-1337473070 tempest-VolumesTestJSON-1337473070-project] 10.222.0.26 "GET /compute/v2.1/os-volumes/detail" status: 200 len: 1033 microversion: 2.1 time: 0.287241 Oct 19 00:48:00.521297 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 719/834] 10.222.0.26 () {58 vars in 1155 bytes} [Wed Oct 19 00:48:00 2022] GET /compute/v2.1/os-volumes/detail => generated 1033 bytes in 291 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 19 00:48:02.841909 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-021c4e1f-94bb-4ed8-adcf-dc80afa0f6a7 tempest-VolumesGetTestJSON-1403677279 tempest-VolumesGetTestJSON-1403677279-project] Action: 'create', calling method: .version_select at 0x7f71cb5e8280>, body: {"volume": {"size": 1, "display_name": "tempest-VolumesGetTestJSON-Volume-1774640920", "metadata": {"Type": "work"}}} {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 19 00:48:03.282757 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-021c4e1f-94bb-4ed8-adcf-dc80afa0f6a7 tempest-VolumesGetTestJSON-1403677279 tempest-VolumesGetTestJSON-1403677279-project] 10.222.0.26 "POST /compute/v2.1/os-volumes" status: 200 len: 353 microversion: 2.1 time: 0.597218 Oct 19 00:48:03.284574 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 720/835] 10.222.0.26 () {60 vars in 1156 bytes} [Wed Oct 19 00:48:02 2022] POST /compute/v2.1/os-volumes => generated 353 bytes in 600 msecs (HTTP/1.1 200) 10 headers in 448 bytes (2 switches on core 0) Oct 19 00:48:03.303576 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-b5679ca2-9229-490a-9998-06529e86ae0b tempest-VolumesGetTestJSON-1403677279 tempest-VolumesGetTestJSON-1403677279-project] Calling method '.version_select at 0x7f71cb5cfa60>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:48:03.378731 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-b5679ca2-9229-490a-9998-06529e86ae0b tempest-VolumesGetTestJSON-1403677279 tempest-VolumesGetTestJSON-1403677279-project] 10.222.0.26 "GET /compute/v2.1/os-volumes/a28ca43d-8f65-4bba-8c4b-c1b404be3c67" status: 200 len: 353 microversion: 2.1 time: 0.083269 Oct 19 00:48:03.379465 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 721/836] 10.222.0.26 () {58 vars in 1245 bytes} [Wed Oct 19 00:48:03 2022] GET /compute/v2.1/os-volumes/a28ca43d-8f65-4bba-8c4b-c1b404be3c67 => generated 353 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 19 00:48:04.397970 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-2df8faa5-87ee-40ed-848f-429fc001e5ee tempest-VolumesGetTestJSON-1403677279 tempest-VolumesGetTestJSON-1403677279-project] Calling method '.version_select at 0x7f71c3fec0d0>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:48:04.474067 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-2df8faa5-87ee-40ed-848f-429fc001e5ee tempest-VolumesGetTestJSON-1403677279 tempest-VolumesGetTestJSON-1403677279-project] 10.222.0.26 "GET /compute/v2.1/os-volumes/a28ca43d-8f65-4bba-8c4b-c1b404be3c67" status: 200 len: 354 microversion: 2.1 time: 0.081507 Oct 19 00:48:04.474997 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 722/837] 10.222.0.26 () {58 vars in 1245 bytes} [Wed Oct 19 00:48:04 2022] GET /compute/v2.1/os-volumes/a28ca43d-8f65-4bba-8c4b-c1b404be3c67 => generated 354 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 19 00:48:04.491020 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-8de24fc5-c7d5-43c5-b112-873f207f3dc7 tempest-VolumesGetTestJSON-1403677279 tempest-VolumesGetTestJSON-1403677279-project] Calling method '.version_select at 0x7f71c3f0e280>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:48:04.575171 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-8de24fc5-c7d5-43c5-b112-873f207f3dc7 tempest-VolumesGetTestJSON-1403677279 tempest-VolumesGetTestJSON-1403677279-project] 10.222.0.26 "GET /compute/v2.1/os-volumes/a28ca43d-8f65-4bba-8c4b-c1b404be3c67" status: 200 len: 354 microversion: 2.1 time: 0.089170 Oct 19 00:48:04.576121 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 723/838] 10.222.0.26 () {58 vars in 1245 bytes} [Wed Oct 19 00:48:04 2022] GET /compute/v2.1/os-volumes/a28ca43d-8f65-4bba-8c4b-c1b404be3c67 => generated 354 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 19 00:48:04.601294 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-9925bc9f-5cba-4ec9-9d22-0254303c473d tempest-VolumesGetTestJSON-1403677279 tempest-VolumesGetTestJSON-1403677279-project] Calling method '.version_select at 0x7f71c3f0e0d0>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:48:04.676759 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-9925bc9f-5cba-4ec9-9d22-0254303c473d tempest-VolumesGetTestJSON-1403677279 tempest-VolumesGetTestJSON-1403677279-project] 10.222.0.26 "DELETE /compute/v2.1/os-volumes/a28ca43d-8f65-4bba-8c4b-c1b404be3c67" status: 202 len: 0 microversion: 2.1 time: 0.079832 Oct 19 00:48:04.677509 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 724/839] 10.222.0.26 () {58 vars in 1248 bytes} [Wed Oct 19 00:48:04 2022] DELETE /compute/v2.1/os-volumes/a28ca43d-8f65-4bba-8c4b-c1b404be3c67 => generated 0 bytes in 81 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Oct 19 00:48:04.687096 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-47ed8811-12d2-4093-811f-5189d4c58edb tempest-VolumesGetTestJSON-1403677279 tempest-VolumesGetTestJSON-1403677279-project] Calling method '.version_select at 0x7f71cb5f03a0>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:48:04.772637 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-47ed8811-12d2-4093-811f-5189d4c58edb tempest-VolumesGetTestJSON-1403677279 tempest-VolumesGetTestJSON-1403677279-project] 10.222.0.26 "GET /compute/v2.1/os-volumes/a28ca43d-8f65-4bba-8c4b-c1b404be3c67" status: 200 len: 353 microversion: 2.1 time: 0.089803 Oct 19 00:48:04.773685 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 725/840] 10.222.0.26 () {58 vars in 1245 bytes} [Wed Oct 19 00:48:04 2022] GET /compute/v2.1/os-volumes/a28ca43d-8f65-4bba-8c4b-c1b404be3c67 => generated 353 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 19 00:48:05.796029 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-049fbb2d-eca2-4592-8113-57a267501a0a tempest-VolumesGetTestJSON-1403677279 tempest-VolumesGetTestJSON-1403677279-project] Calling method '.version_select at 0x7f71cb5e81f0>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:48:05.839310 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.wsgi [None req-049fbb2d-eca2-4592-8113-57a267501a0a tempest-VolumesGetTestJSON-1403677279 tempest-VolumesGetTestJSON-1403677279-project] HTTP exception thrown: Volume a28ca43d-8f65-4bba-8c4b-c1b404be3c67 could not be found. Oct 19 00:48:05.843042 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-049fbb2d-eca2-4592-8113-57a267501a0a tempest-VolumesGetTestJSON-1403677279 tempest-VolumesGetTestJSON-1403677279-project] Request method failure captured: Oct 19 00:48:05.843042 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: request: GET /compute/v2.1/os-volumes/a28ca43d-8f65-4bba-8c4b-c1b404be3c67 HTTP/1.1 Oct 19 00:48:05.843042 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Accept: application/json Oct 19 00:48:05.843042 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Accept-Encoding: identity Oct 19 00:48:05.843042 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Connection: close Oct 19 00:48:05.843042 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Content-Length: 0 Oct 19 00:48:05.843042 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Content-Type: application/json Oct 19 00:48:05.843042 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Host: 10.222.0.26 Oct 19 00:48:05.843042 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Openstack-System-Scope: None Oct 19 00:48:05.843042 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: User-Agent: python-urllib3/1.26.12 Oct 19 00:48:05.843042 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Auth-Token: gAAAAABjT0lCEVOufweirSqOCuV-UKiAhifIqJBIp7bhUCInarBW0SHgClVepQy3Jm60aaP8Y0wQWVuLddKO99UBIhqNmaD08iP1_Lbw_sCylr2jX8eypPKVsH3j2L5IejmLD5_my26QzkNFwJmDbfyot0wPna7SrJvUuVgDU09ioTv64p0f95Q Oct 19 00:48:05.843042 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Domain-Id: None Oct 19 00:48:05.843042 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Domain-Name: None Oct 19 00:48:05.843042 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Identity-Status: Confirmed Oct 19 00:48:05.843042 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Is-Admin-Project: True Oct 19 00:48:05.843042 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Project-Domain-Id: default Oct 19 00:48:05.843042 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Project-Domain-Name: Default Oct 19 00:48:05.843042 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Project-Id: ec9886511380430d8ca93cb3fdb85293 Oct 19 00:48:05.843042 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Project-Name: tempest-VolumesGetTestJSON-1403677279 Oct 19 00:48:05.843042 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Role: member,reader Oct 19 00:48:05.843042 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Roles: member,reader Oct 19 00:48:05.846776 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/ec9886511380430d8ca93cb3fdb85293"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/ec9886511380430d8ca93cb3fdb85293"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/ec9886511380430d8ca93cb3fdb85293"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_ec9886511380430d8ca93cb3fdb85293"}]}] Oct 19 00:48:05.846776 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Tenant: tempest-VolumesGetTestJSON-1403677279 Oct 19 00:48:05.846776 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Tenant-Id: ec9886511380430d8ca93cb3fdb85293 Oct 19 00:48:05.846776 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Tenant-Name: tempest-VolumesGetTestJSON-1403677279 Oct 19 00:48:05.846776 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User: tempest-VolumesGetTestJSON-1403677279-project Oct 19 00:48:05.846776 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User-Domain-Id: default Oct 19 00:48:05.846776 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User-Domain-Name: Default Oct 19 00:48:05.846776 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User-Id: 768941ad9b904febaf4cd2c7b162f799 Oct 19 00:48:05.846776 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User-Name: tempest-VolumesGetTestJSON-1403677279-project Oct 19 00:48:05.846776 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: method: Oct 19 00:48:05.846776 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: exception: Volume a28ca43d-8f65-4bba-8c4b-c1b404be3c67 could not be found. Oct 19 00:48:05.846776 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: action_args: {'id': 'a28ca43d-8f65-4bba-8c4b-c1b404be3c67'} Oct 19 00:48:05.846776 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 19 00:48:05.849169 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 19 00:48:05.849169 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Oct 19 00:48:05.849169 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Oct 19 00:48:05.849169 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Oct 19 00:48:05.849169 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi item = cinderclient( Oct 19 00:48:05.849169 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Oct 19 00:48:05.849169 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Oct 19 00:48:05.849169 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Oct 19 00:48:05.849169 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Oct 19 00:48:05.849169 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Oct 19 00:48:05.849169 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Oct 19 00:48:05.849169 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Oct 19 00:48:05.849169 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Oct 19 00:48:05.849169 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Oct 19 00:48:05.849169 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Oct 19 00:48:05.849169 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume a28ca43d-8f65-4bba-8c4b-c1b404be3c67 could not be found. (HTTP 404) (Request-ID: req-f399df0d-6277-459c-8d28-ea1e87ea4a8c) Oct 19 00:48:05.849169 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:48:05.849169 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 19 00:48:05.849169 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:48:05.851971 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 19 00:48:05.851971 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 120, in show Oct 19 00:48:05.851971 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi vol = self.volume_api.get(context, id) Oct 19 00:48:05.851971 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Oct 19 00:48:05.851971 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Oct 19 00:48:05.851971 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Oct 19 00:48:05.851971 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Oct 19 00:48:05.851971 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Oct 19 00:48:05.851971 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Oct 19 00:48:05.851971 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Oct 19 00:48:05.851971 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Oct 19 00:48:05.851971 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Oct 19 00:48:05.851971 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi item = cinderclient( Oct 19 00:48:05.851971 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Oct 19 00:48:05.851971 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Oct 19 00:48:05.851971 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Oct 19 00:48:05.851971 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Oct 19 00:48:05.851971 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Oct 19 00:48:05.851971 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Oct 19 00:48:05.855642 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Oct 19 00:48:05.855642 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Oct 19 00:48:05.855642 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Oct 19 00:48:05.855642 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Oct 19 00:48:05.855642 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume a28ca43d-8f65-4bba-8c4b-c1b404be3c67 could not be found. Oct 19 00:48:05.855642 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:48:05.855642 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 19 00:48:05.855642 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:48:05.855642 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 19 00:48:05.855642 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 19 00:48:05.855642 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 19 00:48:05.855642 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 19 00:48:05.855642 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 19 00:48:05.855642 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 19 00:48:05.855642 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 19 00:48:05.855642 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 19 00:48:05.855642 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 19 00:48:05.855642 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 122, in show Oct 19 00:48:05.855642 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 19 00:48:05.858753 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume a28ca43d-8f65-4bba-8c4b-c1b404be3c67 could not be found. Oct 19 00:48:05.858753 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:48:05.858753 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 19 00:48:05.858753 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:48:05.858753 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 19 00:48:05.858753 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 19 00:48:05.858753 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 19 00:48:05.858753 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 19 00:48:05.858753 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 19 00:48:05.858753 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume a28ca43d-8f65-4bba-8c4b-c1b404be3c67 could not be found. Oct 19 00:48:05.858753 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:48:05.858753 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-049fbb2d-eca2-4592-8113-57a267501a0a tempest-VolumesGetTestJSON-1403677279 tempest-VolumesGetTestJSON-1403677279-project] Returning 404 to user: Volume a28ca43d-8f65-4bba-8c4b-c1b404be3c67 could not be found. {{(pid=125641) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 19 00:48:05.858753 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-049fbb2d-eca2-4592-8113-57a267501a0a tempest-VolumesGetTestJSON-1403677279 tempest-VolumesGetTestJSON-1403677279-project] 10.222.0.26 "GET /compute/v2.1/os-volumes/a28ca43d-8f65-4bba-8c4b-c1b404be3c67" status: 404 len: 109 microversion: 2.1 time: 0.057873 Oct 19 00:48:05.858753 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 726/841] 10.222.0.26 () {58 vars in 1245 bytes} [Wed Oct 19 00:48:05 2022] GET /compute/v2.1/os-volumes/a28ca43d-8f65-4bba-8c4b-c1b404be3c67 => generated 109 bytes in 61 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Oct 19 00:48:27.380040 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG dbcounter [-] [125641] Writing DB stats nova_api:SELECT=50,nova_api:UPDATE=1,nova_api:DELETE=1 {{(pid=125641) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 00:48:27.381012 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG dbcounter [-] [125641] Writing DB stats nova_cell1:SELECT=85,nova_cell1:UPDATE=2,nova_cell1:INSERT=2 {{(pid=125641) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 00:48:27.387331 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-5f2a812c-fe12-4a76-a4cd-2425be010c1d tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:48:27.387958 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-5f2a812c-fe12-4a76-a4cd-2425be010c1d tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] [instance: b38d4219-8912-4fd9-9d02-7848016f7954] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:48:27.400443 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-5f2a812c-fe12-4a76-a4cd-2425be010c1d tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:48:27.401092 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-5f2a812c-fe12-4a76-a4cd-2425be010c1d tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:48:27.401648 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-5f2a812c-fe12-4a76-a4cd-2425be010c1d tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:48:27.402603 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-5f2a812c-fe12-4a76-a4cd-2425be010c1d tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:48:27.403270 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-5f2a812c-fe12-4a76-a4cd-2425be010c1d tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:48:27.403847 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-5f2a812c-fe12-4a76-a4cd-2425be010c1d tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:48:27.510134 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.objects.instance [None req-5f2a812c-fe12-4a76-a4cd-2425be010c1d tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lazy-loading 'services' on Instance uuid b38d4219-8912-4fd9-9d02-7848016f7954 {{(pid=125641) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 19 00:48:27.617623 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-5f2a812c-fe12-4a76-a4cd-2425be010c1d tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] 10.222.0.26 "DELETE /compute/v2.1/servers/b38d4219-8912-4fd9-9d02-7848016f7954/os-volume_attachments/68f7c32a-29e2-48c4-a946-649b0aa6acf3" status: 202 len: 0 microversion: 2.1 time: 0.238782 Oct 19 00:48:27.618254 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 727/842] 10.222.0.26 () {58 vars in 1416 bytes} [Wed Oct 19 00:48:27 2022] DELETE /compute/v2.1/servers/b38d4219-8912-4fd9-9d02-7848016f7954/os-volume_attachments/68f7c32a-29e2-48c4-a946-649b0aa6acf3 => generated 0 bytes in 241 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Oct 19 00:48:28.835755 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-3600b3a0-a4f0-4371-b62f-765246ea3564 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Action: 'action', calling method: >, body: {"os-stop": {}} {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 19 00:48:28.847136 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-3600b3a0-a4f0-4371-b62f-765246ea3564 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:48:28.847917 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-3600b3a0-a4f0-4371-b62f-765246ea3564 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:48:28.848492 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-3600b3a0-a4f0-4371-b62f-765246ea3564 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:48:28.881089 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-3600b3a0-a4f0-4371-b62f-765246ea3564 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] [instance: b38d4219-8912-4fd9-9d02-7848016f7954] Going to try to stop instance {{(pid=125641) force_stop /opt/stack/nova/nova/compute/api.py:2762}} Oct 19 00:48:28.935894 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-3600b3a0-a4f0-4371-b62f-765246ea3564 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] 10.222.0.26 "POST /compute/v2.1/servers/b38d4219-8912-4fd9-9d02-7848016f7954/action" status: 202 len: 0 microversion: 2.1 time: 0.106003 Oct 19 00:48:28.936515 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 728/843] 10.222.0.26 () {60 vars in 1278 bytes} [Wed Oct 19 00:48:28 2022] POST /compute/v2.1/servers/b38d4219-8912-4fd9-9d02-7848016f7954/action => generated 0 bytes in 108 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Oct 19 00:48:28.948389 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-435ddfb8-7813-4b40-83f6-1ab4bbc9fd05 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:48:28.949581 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-435ddfb8-7813-4b40-83f6-1ab4bbc9fd05 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] [instance: b38d4219-8912-4fd9-9d02-7848016f7954] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:48:28.966520 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-435ddfb8-7813-4b40-83f6-1ab4bbc9fd05 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:48:28.967593 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-435ddfb8-7813-4b40-83f6-1ab4bbc9fd05 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:48:28.968138 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-435ddfb8-7813-4b40-83f6-1ab4bbc9fd05 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:48:28.969716 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-435ddfb8-7813-4b40-83f6-1ab4bbc9fd05 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:48:28.970401 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-435ddfb8-7813-4b40-83f6-1ab4bbc9fd05 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:48:28.971411 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-435ddfb8-7813-4b40-83f6-1ab4bbc9fd05 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:48:29.215353 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-435ddfb8-7813-4b40-83f6-1ab4bbc9fd05 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a48ed2d9ded741ca91afc1fbbd9c608f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '82d7723ef95b4c3faeac9835eeb233a5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:48:29.231756 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-435ddfb8-7813-4b40-83f6-1ab4bbc9fd05 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] 10.222.0.26 "GET /compute/v2.1/servers/b38d4219-8912-4fd9-9d02-7848016f7954" status: 200 len: 1679 microversion: 2.1 time: 0.288958 Oct 19 00:48:29.232450 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 729/844] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:48:28 2022] GET /compute/v2.1/servers/b38d4219-8912-4fd9-9d02-7848016f7954 => generated 1679 bytes in 291 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 19 00:48:30.266721 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-ef7d530f-5d89-4154-988e-91e75d4f119c tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:48:30.269474 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-ef7d530f-5d89-4154-988e-91e75d4f119c tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] [instance: b38d4219-8912-4fd9-9d02-7848016f7954] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:48:30.284530 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ef7d530f-5d89-4154-988e-91e75d4f119c tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:48:30.285281 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ef7d530f-5d89-4154-988e-91e75d4f119c tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:48:30.285905 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ef7d530f-5d89-4154-988e-91e75d4f119c tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:48:30.286895 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ef7d530f-5d89-4154-988e-91e75d4f119c tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:48:30.287498 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ef7d530f-5d89-4154-988e-91e75d4f119c tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:48:30.288090 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ef7d530f-5d89-4154-988e-91e75d4f119c tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:48:30.521378 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-ef7d530f-5d89-4154-988e-91e75d4f119c tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a48ed2d9ded741ca91afc1fbbd9c608f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '82d7723ef95b4c3faeac9835eeb233a5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:48:30.536997 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-ef7d530f-5d89-4154-988e-91e75d4f119c tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] 10.222.0.26 "GET /compute/v2.1/servers/b38d4219-8912-4fd9-9d02-7848016f7954" status: 200 len: 1679 microversion: 2.1 time: 0.279325 Oct 19 00:48:30.537997 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 730/845] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:48:30 2022] GET /compute/v2.1/servers/b38d4219-8912-4fd9-9d02-7848016f7954 => generated 1679 bytes in 282 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 19 00:48:31.578508 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-ffada3f5-02de-46f6-bb41-3ebbdf20d67c tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:48:31.579292 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-ffada3f5-02de-46f6-bb41-3ebbdf20d67c tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] [instance: b38d4219-8912-4fd9-9d02-7848016f7954] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:48:31.597465 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ffada3f5-02de-46f6-bb41-3ebbdf20d67c tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:48:31.598063 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ffada3f5-02de-46f6-bb41-3ebbdf20d67c tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:48:31.598717 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ffada3f5-02de-46f6-bb41-3ebbdf20d67c tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:48:31.600554 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ffada3f5-02de-46f6-bb41-3ebbdf20d67c tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:48:31.600554 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ffada3f5-02de-46f6-bb41-3ebbdf20d67c tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:48:31.601220 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ffada3f5-02de-46f6-bb41-3ebbdf20d67c tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:48:31.824636 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-ffada3f5-02de-46f6-bb41-3ebbdf20d67c tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a48ed2d9ded741ca91afc1fbbd9c608f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '82d7723ef95b4c3faeac9835eeb233a5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:48:31.835505 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-ffada3f5-02de-46f6-bb41-3ebbdf20d67c tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] 10.222.0.26 "GET /compute/v2.1/servers/b38d4219-8912-4fd9-9d02-7848016f7954" status: 200 len: 1679 microversion: 2.1 time: 0.262692 Oct 19 00:48:31.836190 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 731/846] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:48:31 2022] GET /compute/v2.1/servers/b38d4219-8912-4fd9-9d02-7848016f7954 => generated 1679 bytes in 264 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:48:32.861090 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-36414b7f-8ea5-4963-87d3-351900cd9e76 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:48:32.863186 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-36414b7f-8ea5-4963-87d3-351900cd9e76 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] [instance: b38d4219-8912-4fd9-9d02-7848016f7954] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:48:32.874781 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-36414b7f-8ea5-4963-87d3-351900cd9e76 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:48:32.875371 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-36414b7f-8ea5-4963-87d3-351900cd9e76 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:48:32.875998 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-36414b7f-8ea5-4963-87d3-351900cd9e76 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:48:32.880039 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-36414b7f-8ea5-4963-87d3-351900cd9e76 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:48:32.880848 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-36414b7f-8ea5-4963-87d3-351900cd9e76 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:48:32.881578 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-36414b7f-8ea5-4963-87d3-351900cd9e76 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:48:33.084514 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-36414b7f-8ea5-4963-87d3-351900cd9e76 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a48ed2d9ded741ca91afc1fbbd9c608f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '82d7723ef95b4c3faeac9835eeb233a5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:48:33.096727 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-36414b7f-8ea5-4963-87d3-351900cd9e76 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] 10.222.0.26 "GET /compute/v2.1/servers/b38d4219-8912-4fd9-9d02-7848016f7954" status: 200 len: 1679 microversion: 2.1 time: 0.240296 Oct 19 00:48:33.097929 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 732/847] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:48:32 2022] GET /compute/v2.1/servers/b38d4219-8912-4fd9-9d02-7848016f7954 => generated 1679 bytes in 242 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 19 00:48:34.124658 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-66896e3f-8bc8-43f4-8b91-3be38342a9a9 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:48:34.126213 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-66896e3f-8bc8-43f4-8b91-3be38342a9a9 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] [instance: b38d4219-8912-4fd9-9d02-7848016f7954] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:48:34.136141 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-66896e3f-8bc8-43f4-8b91-3be38342a9a9 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:48:34.136915 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-66896e3f-8bc8-43f4-8b91-3be38342a9a9 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:48:34.137633 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-66896e3f-8bc8-43f4-8b91-3be38342a9a9 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:48:34.138382 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-66896e3f-8bc8-43f4-8b91-3be38342a9a9 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:48:34.139228 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-66896e3f-8bc8-43f4-8b91-3be38342a9a9 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:48:34.139730 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-66896e3f-8bc8-43f4-8b91-3be38342a9a9 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:48:34.360739 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-66896e3f-8bc8-43f4-8b91-3be38342a9a9 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a48ed2d9ded741ca91afc1fbbd9c608f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '82d7723ef95b4c3faeac9835eeb233a5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:48:34.375425 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-66896e3f-8bc8-43f4-8b91-3be38342a9a9 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] 10.222.0.26 "GET /compute/v2.1/servers/b38d4219-8912-4fd9-9d02-7848016f7954" status: 200 len: 1679 microversion: 2.1 time: 0.255507 Oct 19 00:48:34.376926 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 733/848] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:48:34 2022] GET /compute/v2.1/servers/b38d4219-8912-4fd9-9d02-7848016f7954 => generated 1679 bytes in 258 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:48:35.406840 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-4e3865c5-cc57-456d-a0f0-72fe3d94b4f7 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:48:35.408278 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-4e3865c5-cc57-456d-a0f0-72fe3d94b4f7 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] [instance: b38d4219-8912-4fd9-9d02-7848016f7954] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:48:35.415840 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-4e3865c5-cc57-456d-a0f0-72fe3d94b4f7 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:48:35.416319 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-4e3865c5-cc57-456d-a0f0-72fe3d94b4f7 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:48:35.416758 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-4e3865c5-cc57-456d-a0f0-72fe3d94b4f7 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:48:35.417505 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-4e3865c5-cc57-456d-a0f0-72fe3d94b4f7 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:48:35.417973 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-4e3865c5-cc57-456d-a0f0-72fe3d94b4f7 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:48:35.418379 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-4e3865c5-cc57-456d-a0f0-72fe3d94b4f7 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:48:35.659569 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-4e3865c5-cc57-456d-a0f0-72fe3d94b4f7 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a48ed2d9ded741ca91afc1fbbd9c608f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '82d7723ef95b4c3faeac9835eeb233a5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:48:35.667809 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-4e3865c5-cc57-456d-a0f0-72fe3d94b4f7 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] 10.222.0.26 "GET /compute/v2.1/servers/b38d4219-8912-4fd9-9d02-7848016f7954" status: 200 len: 1679 microversion: 2.1 time: 0.265580 Oct 19 00:48:35.668398 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 734/849] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:48:35 2022] GET /compute/v2.1/servers/b38d4219-8912-4fd9-9d02-7848016f7954 => generated 1679 bytes in 267 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 19 00:48:36.698424 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-b0fd28e0-0099-4da6-9c19-756dceb55473 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:48:36.700760 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-b0fd28e0-0099-4da6-9c19-756dceb55473 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] [instance: b38d4219-8912-4fd9-9d02-7848016f7954] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:48:36.712546 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-b0fd28e0-0099-4da6-9c19-756dceb55473 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:48:36.713416 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-b0fd28e0-0099-4da6-9c19-756dceb55473 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:48:36.714152 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-b0fd28e0-0099-4da6-9c19-756dceb55473 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:48:36.715250 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-b0fd28e0-0099-4da6-9c19-756dceb55473 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:48:36.715985 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-b0fd28e0-0099-4da6-9c19-756dceb55473 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:48:36.716656 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-b0fd28e0-0099-4da6-9c19-756dceb55473 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:48:36.924354 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-b0fd28e0-0099-4da6-9c19-756dceb55473 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a48ed2d9ded741ca91afc1fbbd9c608f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '82d7723ef95b4c3faeac9835eeb233a5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:48:36.938507 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-b0fd28e0-0099-4da6-9c19-756dceb55473 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] 10.222.0.26 "GET /compute/v2.1/servers/b38d4219-8912-4fd9-9d02-7848016f7954" status: 200 len: 1679 microversion: 2.1 time: 0.247731 Oct 19 00:48:36.939491 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 735/850] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:48:36 2022] GET /compute/v2.1/servers/b38d4219-8912-4fd9-9d02-7848016f7954 => generated 1679 bytes in 251 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 19 00:48:37.975951 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-c8f6adbe-393b-4354-9462-f45b89946d9f tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:48:37.976707 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-c8f6adbe-393b-4354-9462-f45b89946d9f tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] [instance: b38d4219-8912-4fd9-9d02-7848016f7954] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:48:37.984935 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-c8f6adbe-393b-4354-9462-f45b89946d9f tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:48:37.985577 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-c8f6adbe-393b-4354-9462-f45b89946d9f tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:48:37.986765 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-c8f6adbe-393b-4354-9462-f45b89946d9f tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:48:37.987857 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-c8f6adbe-393b-4354-9462-f45b89946d9f tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:48:37.988426 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-c8f6adbe-393b-4354-9462-f45b89946d9f tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:48:37.988953 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-c8f6adbe-393b-4354-9462-f45b89946d9f tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:48:38.146900 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-c8f6adbe-393b-4354-9462-f45b89946d9f tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a48ed2d9ded741ca91afc1fbbd9c608f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '82d7723ef95b4c3faeac9835eeb233a5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:48:38.159599 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-c8f6adbe-393b-4354-9462-f45b89946d9f tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] 10.222.0.26 "GET /compute/v2.1/servers/b38d4219-8912-4fd9-9d02-7848016f7954" status: 200 len: 1679 microversion: 2.1 time: 0.187622 Oct 19 00:48:38.160400 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 736/851] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:48:37 2022] GET /compute/v2.1/servers/b38d4219-8912-4fd9-9d02-7848016f7954 => generated 1679 bytes in 190 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 19 00:48:39.200698 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-9aab9e75-56aa-4fa2-b7c0-8336af8be1d3 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:48:39.201911 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-9aab9e75-56aa-4fa2-b7c0-8336af8be1d3 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] [instance: b38d4219-8912-4fd9-9d02-7848016f7954] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:48:39.210989 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-9aab9e75-56aa-4fa2-b7c0-8336af8be1d3 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:48:39.211489 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-9aab9e75-56aa-4fa2-b7c0-8336af8be1d3 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:48:39.211930 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-9aab9e75-56aa-4fa2-b7c0-8336af8be1d3 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:48:39.212665 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-9aab9e75-56aa-4fa2-b7c0-8336af8be1d3 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:48:39.213122 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-9aab9e75-56aa-4fa2-b7c0-8336af8be1d3 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:48:39.213544 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-9aab9e75-56aa-4fa2-b7c0-8336af8be1d3 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:48:39.390715 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-9aab9e75-56aa-4fa2-b7c0-8336af8be1d3 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a48ed2d9ded741ca91afc1fbbd9c608f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '82d7723ef95b4c3faeac9835eeb233a5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:48:39.404456 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-9aab9e75-56aa-4fa2-b7c0-8336af8be1d3 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] 10.222.0.26 "GET /compute/v2.1/servers/b38d4219-8912-4fd9-9d02-7848016f7954" status: 200 len: 1679 microversion: 2.1 time: 0.208817 Oct 19 00:48:39.405545 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 737/852] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:48:39 2022] GET /compute/v2.1/servers/b38d4219-8912-4fd9-9d02-7848016f7954 => generated 1679 bytes in 211 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 19 00:48:40.449647 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-cb24a0af-a55b-4b9b-9aa2-dab12b2a0d2a tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:48:40.450926 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-cb24a0af-a55b-4b9b-9aa2-dab12b2a0d2a tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] [instance: b38d4219-8912-4fd9-9d02-7848016f7954] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:48:40.460029 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-cb24a0af-a55b-4b9b-9aa2-dab12b2a0d2a tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:48:40.460632 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-cb24a0af-a55b-4b9b-9aa2-dab12b2a0d2a tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:48:40.461585 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-cb24a0af-a55b-4b9b-9aa2-dab12b2a0d2a tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:48:40.462711 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-cb24a0af-a55b-4b9b-9aa2-dab12b2a0d2a tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:48:40.463420 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-cb24a0af-a55b-4b9b-9aa2-dab12b2a0d2a tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:48:40.464128 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-cb24a0af-a55b-4b9b-9aa2-dab12b2a0d2a tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:48:40.623928 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-cb24a0af-a55b-4b9b-9aa2-dab12b2a0d2a tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a48ed2d9ded741ca91afc1fbbd9c608f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '82d7723ef95b4c3faeac9835eeb233a5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:48:40.632816 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-cb24a0af-a55b-4b9b-9aa2-dab12b2a0d2a tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] 10.222.0.26 "GET /compute/v2.1/servers/b38d4219-8912-4fd9-9d02-7848016f7954" status: 200 len: 1656 microversion: 2.1 time: 0.191249 Oct 19 00:48:40.633406 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 738/853] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:48:40 2022] GET /compute/v2.1/servers/b38d4219-8912-4fd9-9d02-7848016f7954 => generated 1656 bytes in 193 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:48:40.656577 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-1a16cfc4-256d-40f8-9a0f-04619559a21f tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Action: 'action', calling method: >, body: {"os-start": {}} {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 19 00:48:40.663445 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-1a16cfc4-256d-40f8-9a0f-04619559a21f tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:48:40.664000 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-1a16cfc4-256d-40f8-9a0f-04619559a21f tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:48:40.664485 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-1a16cfc4-256d-40f8-9a0f-04619559a21f tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:48:40.697307 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-1a16cfc4-256d-40f8-9a0f-04619559a21f tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] [instance: b38d4219-8912-4fd9-9d02-7848016f7954] Going to try to start instance {{(pid=125641) start /opt/stack/nova/nova/compute/api.py:2785}} Oct 19 00:48:40.749973 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-1a16cfc4-256d-40f8-9a0f-04619559a21f tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] 10.222.0.26 "POST /compute/v2.1/servers/b38d4219-8912-4fd9-9d02-7848016f7954/action" status: 202 len: 0 microversion: 2.1 time: 0.097129 Oct 19 00:48:40.750788 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 739/854] 10.222.0.26 () {60 vars in 1278 bytes} [Wed Oct 19 00:48:40 2022] POST /compute/v2.1/servers/b38d4219-8912-4fd9-9d02-7848016f7954/action => generated 0 bytes in 99 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Oct 19 00:48:40.762182 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-838da9d4-dc96-4d22-9c74-09b08750d905 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:48:40.763173 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-838da9d4-dc96-4d22-9c74-09b08750d905 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] [instance: b38d4219-8912-4fd9-9d02-7848016f7954] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:48:40.771607 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-838da9d4-dc96-4d22-9c74-09b08750d905 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:48:40.772029 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-838da9d4-dc96-4d22-9c74-09b08750d905 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:48:40.772473 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-838da9d4-dc96-4d22-9c74-09b08750d905 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:48:40.773320 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-838da9d4-dc96-4d22-9c74-09b08750d905 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:48:40.773796 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-838da9d4-dc96-4d22-9c74-09b08750d905 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:48:40.774233 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-838da9d4-dc96-4d22-9c74-09b08750d905 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:48:40.963997 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-838da9d4-dc96-4d22-9c74-09b08750d905 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a48ed2d9ded741ca91afc1fbbd9c608f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '82d7723ef95b4c3faeac9835eeb233a5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:48:40.971769 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG dbcounter [-] [125641] Writing DB stats nova_cell1:SELECT=95,nova_cell1:INSERT=3,nova_cell1:UPDATE=2 {{(pid=125641) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 00:48:40.981759 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-838da9d4-dc96-4d22-9c74-09b08750d905 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] 10.222.0.26 "GET /compute/v2.1/servers/b38d4219-8912-4fd9-9d02-7848016f7954" status: 200 len: 1665 microversion: 2.1 time: 0.224526 Oct 19 00:48:40.982831 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 740/855] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:48:40 2022] GET /compute/v2.1/servers/b38d4219-8912-4fd9-9d02-7848016f7954 => generated 1665 bytes in 227 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 19 00:48:42.010909 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-78e45ff5-5d78-4b38-92e1-730635946100 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:48:42.011990 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-78e45ff5-5d78-4b38-92e1-730635946100 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] [instance: b38d4219-8912-4fd9-9d02-7848016f7954] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:48:42.025763 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-78e45ff5-5d78-4b38-92e1-730635946100 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:48:42.026344 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-78e45ff5-5d78-4b38-92e1-730635946100 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:48:42.026762 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-78e45ff5-5d78-4b38-92e1-730635946100 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:48:42.027572 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-78e45ff5-5d78-4b38-92e1-730635946100 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:48:42.028017 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-78e45ff5-5d78-4b38-92e1-730635946100 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:48:42.028449 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-78e45ff5-5d78-4b38-92e1-730635946100 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:48:42.275207 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-78e45ff5-5d78-4b38-92e1-730635946100 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a48ed2d9ded741ca91afc1fbbd9c608f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '82d7723ef95b4c3faeac9835eeb233a5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:48:42.288733 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-78e45ff5-5d78-4b38-92e1-730635946100 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] 10.222.0.26 "GET /compute/v2.1/servers/b38d4219-8912-4fd9-9d02-7848016f7954" status: 200 len: 1665 microversion: 2.1 time: 0.284261 Oct 19 00:48:42.289336 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 741/856] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:48:42 2022] GET /compute/v2.1/servers/b38d4219-8912-4fd9-9d02-7848016f7954 => generated 1665 bytes in 286 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 19 00:48:43.317448 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-cf5913bb-1e58-4c50-851a-82cba9cbb78e tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:48:43.318617 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-cf5913bb-1e58-4c50-851a-82cba9cbb78e tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] [instance: b38d4219-8912-4fd9-9d02-7848016f7954] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:48:43.326675 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-cf5913bb-1e58-4c50-851a-82cba9cbb78e tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:48:43.327361 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-cf5913bb-1e58-4c50-851a-82cba9cbb78e tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:48:43.328015 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-cf5913bb-1e58-4c50-851a-82cba9cbb78e tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:48:43.328973 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-cf5913bb-1e58-4c50-851a-82cba9cbb78e tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:48:43.330121 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-cf5913bb-1e58-4c50-851a-82cba9cbb78e tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:48:43.330757 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-cf5913bb-1e58-4c50-851a-82cba9cbb78e tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:48:43.532697 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-cf5913bb-1e58-4c50-851a-82cba9cbb78e tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a48ed2d9ded741ca91afc1fbbd9c608f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '82d7723ef95b4c3faeac9835eeb233a5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:48:43.552544 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-cf5913bb-1e58-4c50-851a-82cba9cbb78e tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] 10.222.0.26 "GET /compute/v2.1/servers/b38d4219-8912-4fd9-9d02-7848016f7954" status: 200 len: 1669 microversion: 2.1 time: 0.240456 Oct 19 00:48:43.554151 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 742/857] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:48:43 2022] GET /compute/v2.1/servers/b38d4219-8912-4fd9-9d02-7848016f7954 => generated 1669 bytes in 243 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 19 00:48:57.821527 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-a15f2d8e-da69-4ed8-857e-23b3b9629f4f tempest-VolumesNegativeTest-969857559 tempest-VolumesNegativeTest-969857559-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesNegativeTest-instance-1399020478", "imageRef": "b387b4f8-f75e-4a68-b49c-a015bf61c46b", "flavorRef": "452"}} {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 19 00:48:57.829313 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG dbcounter [-] [125641] Writing DB stats nova_api:SELECT=32 {{(pid=125641) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 00:48:57.829981 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG dbcounter [-] [125641] Writing DB stats nova_cell1:SELECT=12 {{(pid=125641) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 00:48:57.882369 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.network.neutron [None req-a15f2d8e-da69-4ed8-857e-23b3b9629f4f tempest-VolumesNegativeTest-969857559 tempest-VolumesNegativeTest-969857559-project] validate_networks() for None {{(pid=125641) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 19 00:48:58.503894 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.virt.hardware [None req-a15f2d8e-da69-4ed8-857e-23b3b9629f4f tempest-VolumesNegativeTest-969857559 tempest-VolumesNegativeTest-969857559-project] Flavor limits 0:0:0 {{(pid=125641) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 19 00:48:58.504311 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.virt.hardware [None req-a15f2d8e-da69-4ed8-857e-23b3b9629f4f tempest-VolumesNegativeTest-969857559 tempest-VolumesNegativeTest-969857559-project] Image limits 0:0:0 {{(pid=125641) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 19 00:48:58.504781 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.virt.hardware [None req-a15f2d8e-da69-4ed8-857e-23b3b9629f4f tempest-VolumesNegativeTest-969857559 tempest-VolumesNegativeTest-969857559-project] Flavor pref 0:0:0 {{(pid=125641) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 19 00:48:58.505182 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.virt.hardware [None req-a15f2d8e-da69-4ed8-857e-23b3b9629f4f tempest-VolumesNegativeTest-969857559 tempest-VolumesNegativeTest-969857559-project] Image pref 0:0:0 {{(pid=125641) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 19 00:48:58.505582 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.virt.hardware [None req-a15f2d8e-da69-4ed8-857e-23b3b9629f4f tempest-VolumesNegativeTest-969857559 tempest-VolumesNegativeTest-969857559-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=125641) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 19 00:48:58.516323 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-a15f2d8e-da69-4ed8-857e-23b3b9629f4f tempest-VolumesNegativeTest-969857559 tempest-VolumesNegativeTest-969857559-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:48:58.516862 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-a15f2d8e-da69-4ed8-857e-23b3b9629f4f tempest-VolumesNegativeTest-969857559 tempest-VolumesNegativeTest-969857559-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:48:58.517294 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-a15f2d8e-da69-4ed8-857e-23b3b9629f4f tempest-VolumesNegativeTest-969857559 tempest-VolumesNegativeTest-969857559-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:48:58.518159 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-a15f2d8e-da69-4ed8-857e-23b3b9629f4f tempest-VolumesNegativeTest-969857559 tempest-VolumesNegativeTest-969857559-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:48:58.518707 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-a15f2d8e-da69-4ed8-857e-23b3b9629f4f tempest-VolumesNegativeTest-969857559 tempest-VolumesNegativeTest-969857559-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:48:58.519134 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-a15f2d8e-da69-4ed8-857e-23b3b9629f4f tempest-VolumesNegativeTest-969857559 tempest-VolumesNegativeTest-969857559-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:48:58.537867 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.quota [None req-a15f2d8e-da69-4ed8-857e-23b3b9629f4f tempest-VolumesNegativeTest-969857559 tempest-VolumesNegativeTest-969857559-project] Getting quotas for project 2a1efe09c74b46f39c467c6a602abd25. Resources: {'cores', 'ram', 'instances'} {{(pid=125641) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 19 00:48:58.545965 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.quota [None req-a15f2d8e-da69-4ed8-857e-23b3b9629f4f tempest-VolumesNegativeTest-969857559 tempest-VolumesNegativeTest-969857559-project] Getting quotas for user 796ecf06911540789ec035bc41d52031 and project 2a1efe09c74b46f39c467c6a602abd25. Resources: {'cores', 'ram', 'instances'} {{(pid=125641) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 19 00:48:58.563013 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-a15f2d8e-da69-4ed8-857e-23b3b9629f4f tempest-VolumesNegativeTest-969857559 tempest-VolumesNegativeTest-969857559-project] Going to run 1 instances... {{(pid=125641) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 19 00:48:58.565547 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-a15f2d8e-da69-4ed8-857e-23b3b9629f4f tempest-VolumesNegativeTest-969857559 tempest-VolumesNegativeTest-969857559-project] [instance: 813d51c8-3f4b-4caf-ac65-1c18cca506f7] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='b387b4f8-f75e-4a68-b49c-a015bf61c46b',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=125641) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 19 00:48:58.621060 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-a15f2d8e-da69-4ed8-857e-23b3b9629f4f tempest-VolumesNegativeTest-969857559 tempest-VolumesNegativeTest-969857559-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.804535 Oct 19 00:48:58.621738 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 743/858] 10.222.0.26 () {60 vars in 1147 bytes} [Wed Oct 19 00:48:57 2022] POST /compute/v2.1/servers => generated 380 bytes in 806 msecs (HTTP/1.1 202) 10 headers in 451 bytes (2 switches on core 0) Oct 19 00:48:58.633771 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-b5d06881-6011-409d-a0f8-41f241d20fc8 tempest-VolumesNegativeTest-969857559 tempest-VolumesNegativeTest-969857559-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:48:58.634394 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-b5d06881-6011-409d-a0f8-41f241d20fc8 tempest-VolumesNegativeTest-969857559 tempest-VolumesNegativeTest-969857559-project] [instance: 813d51c8-3f4b-4caf-ac65-1c18cca506f7] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:48:58.696220 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-b5d06881-6011-409d-a0f8-41f241d20fc8 tempest-VolumesNegativeTest-969857559 tempest-VolumesNegativeTest-969857559-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '796ecf06911540789ec035bc41d52031', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a1efe09c74b46f39c467c6a602abd25', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:48:58.710532 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-b5d06881-6011-409d-a0f8-41f241d20fc8 tempest-VolumesNegativeTest-969857559 tempest-VolumesNegativeTest-969857559-project] 10.222.0.26 "GET /compute/v2.1/servers/813d51c8-3f4b-4caf-ac65-1c18cca506f7" status: 200 len: 1215 microversion: 2.1 time: 0.080191 Oct 19 00:48:58.711607 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 744/859] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:48:58 2022] GET /compute/v2.1/servers/813d51c8-3f4b-4caf-ac65-1c18cca506f7 => generated 1215 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 19 00:48:59.736604 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-82e9dec0-eddd-4982-8090-e63068ec2bf9 tempest-VolumesNegativeTest-969857559 tempest-VolumesNegativeTest-969857559-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:48:59.737338 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-82e9dec0-eddd-4982-8090-e63068ec2bf9 tempest-VolumesNegativeTest-969857559 tempest-VolumesNegativeTest-969857559-project] [instance: 813d51c8-3f4b-4caf-ac65-1c18cca506f7] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:48:59.745212 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-82e9dec0-eddd-4982-8090-e63068ec2bf9 tempest-VolumesNegativeTest-969857559 tempest-VolumesNegativeTest-969857559-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:48:59.745714 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-82e9dec0-eddd-4982-8090-e63068ec2bf9 tempest-VolumesNegativeTest-969857559 tempest-VolumesNegativeTest-969857559-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:48:59.746161 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-82e9dec0-eddd-4982-8090-e63068ec2bf9 tempest-VolumesNegativeTest-969857559 tempest-VolumesNegativeTest-969857559-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:48:59.746944 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-82e9dec0-eddd-4982-8090-e63068ec2bf9 tempest-VolumesNegativeTest-969857559 tempest-VolumesNegativeTest-969857559-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:48:59.747407 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-82e9dec0-eddd-4982-8090-e63068ec2bf9 tempest-VolumesNegativeTest-969857559 tempest-VolumesNegativeTest-969857559-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:48:59.747838 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-82e9dec0-eddd-4982-8090-e63068ec2bf9 tempest-VolumesNegativeTest-969857559 tempest-VolumesNegativeTest-969857559-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:48:59.819083 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-82e9dec0-eddd-4982-8090-e63068ec2bf9 tempest-VolumesNegativeTest-969857559 tempest-VolumesNegativeTest-969857559-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '796ecf06911540789ec035bc41d52031', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a1efe09c74b46f39c467c6a602abd25', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:48:59.827739 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-82e9dec0-eddd-4982-8090-e63068ec2bf9 tempest-VolumesNegativeTest-969857559 tempest-VolumesNegativeTest-969857559-project] 10.222.0.26 "GET /compute/v2.1/servers/813d51c8-3f4b-4caf-ac65-1c18cca506f7" status: 200 len: 1267 microversion: 2.1 time: 0.095739 Oct 19 00:48:59.828398 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 745/860] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:48:59 2022] GET /compute/v2.1/servers/813d51c8-3f4b-4caf-ac65-1c18cca506f7 => generated 1267 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 19 00:49:00.852801 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-55479be1-07e7-467e-a554-efd9039eb359 tempest-VolumesNegativeTest-969857559 tempest-VolumesNegativeTest-969857559-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:49:00.854190 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-55479be1-07e7-467e-a554-efd9039eb359 tempest-VolumesNegativeTest-969857559 tempest-VolumesNegativeTest-969857559-project] [instance: 813d51c8-3f4b-4caf-ac65-1c18cca506f7] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:49:00.863168 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-55479be1-07e7-467e-a554-efd9039eb359 tempest-VolumesNegativeTest-969857559 tempest-VolumesNegativeTest-969857559-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:49:00.863691 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-55479be1-07e7-467e-a554-efd9039eb359 tempest-VolumesNegativeTest-969857559 tempest-VolumesNegativeTest-969857559-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:49:00.864132 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-55479be1-07e7-467e-a554-efd9039eb359 tempest-VolumesNegativeTest-969857559 tempest-VolumesNegativeTest-969857559-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:49:00.864864 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-55479be1-07e7-467e-a554-efd9039eb359 tempest-VolumesNegativeTest-969857559 tempest-VolumesNegativeTest-969857559-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:49:00.865321 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-55479be1-07e7-467e-a554-efd9039eb359 tempest-VolumesNegativeTest-969857559 tempest-VolumesNegativeTest-969857559-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:49:00.865748 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-55479be1-07e7-467e-a554-efd9039eb359 tempest-VolumesNegativeTest-969857559 tempest-VolumesNegativeTest-969857559-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:49:00.941625 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-55479be1-07e7-467e-a554-efd9039eb359 tempest-VolumesNegativeTest-969857559 tempest-VolumesNegativeTest-969857559-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '796ecf06911540789ec035bc41d52031', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a1efe09c74b46f39c467c6a602abd25', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:49:00.953230 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-55479be1-07e7-467e-a554-efd9039eb359 tempest-VolumesNegativeTest-969857559 tempest-VolumesNegativeTest-969857559-project] 10.222.0.26 "GET /compute/v2.1/servers/813d51c8-3f4b-4caf-ac65-1c18cca506f7" status: 200 len: 1273 microversion: 2.1 time: 0.104450 Oct 19 00:49:00.954202 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 746/861] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:49:00 2022] GET /compute/v2.1/servers/813d51c8-3f4b-4caf-ac65-1c18cca506f7 => generated 1273 bytes in 106 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 19 00:49:01.991109 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-12e22059-542e-4f89-96b8-eb993b780c7b tempest-VolumesNegativeTest-969857559 tempest-VolumesNegativeTest-969857559-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:49:01.992537 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-12e22059-542e-4f89-96b8-eb993b780c7b tempest-VolumesNegativeTest-969857559 tempest-VolumesNegativeTest-969857559-project] [instance: 813d51c8-3f4b-4caf-ac65-1c18cca506f7] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:49:01.999744 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-12e22059-542e-4f89-96b8-eb993b780c7b tempest-VolumesNegativeTest-969857559 tempest-VolumesNegativeTest-969857559-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:49:02.000244 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-12e22059-542e-4f89-96b8-eb993b780c7b tempest-VolumesNegativeTest-969857559 tempest-VolumesNegativeTest-969857559-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:49:02.000875 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-12e22059-542e-4f89-96b8-eb993b780c7b tempest-VolumesNegativeTest-969857559 tempest-VolumesNegativeTest-969857559-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:49:02.001697 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-12e22059-542e-4f89-96b8-eb993b780c7b tempest-VolumesNegativeTest-969857559 tempest-VolumesNegativeTest-969857559-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:49:02.002167 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-12e22059-542e-4f89-96b8-eb993b780c7b tempest-VolumesNegativeTest-969857559 tempest-VolumesNegativeTest-969857559-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:49:02.002712 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-12e22059-542e-4f89-96b8-eb993b780c7b tempest-VolumesNegativeTest-969857559 tempest-VolumesNegativeTest-969857559-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:49:02.161342 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-12e22059-542e-4f89-96b8-eb993b780c7b tempest-VolumesNegativeTest-969857559 tempest-VolumesNegativeTest-969857559-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '796ecf06911540789ec035bc41d52031', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a1efe09c74b46f39c467c6a602abd25', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:49:02.177953 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-12e22059-542e-4f89-96b8-eb993b780c7b tempest-VolumesNegativeTest-969857559 tempest-VolumesNegativeTest-969857559-project] 10.222.0.26 "GET /compute/v2.1/servers/813d51c8-3f4b-4caf-ac65-1c18cca506f7" status: 200 len: 1315 microversion: 2.1 time: 0.190326 Oct 19 00:49:02.179245 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 747/862] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:49:01 2022] GET /compute/v2.1/servers/813d51c8-3f4b-4caf-ac65-1c18cca506f7 => generated 1315 bytes in 193 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 19 00:49:02.317360 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [req-e416e9ec-245a-44d4-966c-8527a24f7f03 req-f317199f-24f4-47b2-a670-5c84e262ae02 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "813d51c8-3f4b-4caf-ac65-1c18cca506f7", "tag": "1c927b91-ee97-413c-b510-50f66cc4fd5c"}]} {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 19 00:49:02.340749 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-e416e9ec-245a-44d4-966c-8527a24f7f03 req-f317199f-24f4-47b2-a670-5c84e262ae02 service nova] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:49:02.341409 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-e416e9ec-245a-44d4-966c-8527a24f7f03 req-f317199f-24f4-47b2-a670-5c84e262ae02 service nova] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:49:02.341856 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-e416e9ec-245a-44d4-966c-8527a24f7f03 req-f317199f-24f4-47b2-a670-5c84e262ae02 service nova] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:49:02.360695 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.compute.server_external_events [req-e416e9ec-245a-44d4-966c-8527a24f7f03 req-f317199f-24f4-47b2-a670-5c84e262ae02 service nova] Creating event network-changed:1c927b91-ee97-413c-b510-50f66cc4fd5c for instance 813d51c8-3f4b-4caf-ac65-1c18cca506f7 on hv-cinder-maste Oct 19 00:49:02.371703 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [req-e416e9ec-245a-44d4-966c-8527a24f7f03 req-f317199f-24f4-47b2-a670-5c84e262ae02 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.065598 Oct 19 00:49:02.373395 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 748/863] 10.222.0.26 () {64 vars in 1317 bytes} [Wed Oct 19 00:49:02 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 70 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 19 00:49:03.205528 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-6d95fe21-f474-47ac-ade9-86476337a4f6 tempest-VolumesNegativeTest-969857559 tempest-VolumesNegativeTest-969857559-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:49:03.206325 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-6d95fe21-f474-47ac-ade9-86476337a4f6 tempest-VolumesNegativeTest-969857559 tempest-VolumesNegativeTest-969857559-project] [instance: 813d51c8-3f4b-4caf-ac65-1c18cca506f7] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:49:03.213399 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-6d95fe21-f474-47ac-ade9-86476337a4f6 tempest-VolumesNegativeTest-969857559 tempest-VolumesNegativeTest-969857559-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:49:03.213885 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-6d95fe21-f474-47ac-ade9-86476337a4f6 tempest-VolumesNegativeTest-969857559 tempest-VolumesNegativeTest-969857559-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:49:03.214319 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-6d95fe21-f474-47ac-ade9-86476337a4f6 tempest-VolumesNegativeTest-969857559 tempest-VolumesNegativeTest-969857559-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:49:03.215060 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-6d95fe21-f474-47ac-ade9-86476337a4f6 tempest-VolumesNegativeTest-969857559 tempest-VolumesNegativeTest-969857559-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:49:03.215518 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-6d95fe21-f474-47ac-ade9-86476337a4f6 tempest-VolumesNegativeTest-969857559 tempest-VolumesNegativeTest-969857559-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:49:03.215922 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-6d95fe21-f474-47ac-ade9-86476337a4f6 tempest-VolumesNegativeTest-969857559 tempest-VolumesNegativeTest-969857559-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:49:03.364010 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-6d95fe21-f474-47ac-ade9-86476337a4f6 tempest-VolumesNegativeTest-969857559 tempest-VolumesNegativeTest-969857559-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '796ecf06911540789ec035bc41d52031', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a1efe09c74b46f39c467c6a602abd25', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:49:03.373664 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-6d95fe21-f474-47ac-ade9-86476337a4f6 tempest-VolumesNegativeTest-969857559 tempest-VolumesNegativeTest-969857559-project] 10.222.0.26 "GET /compute/v2.1/servers/813d51c8-3f4b-4caf-ac65-1c18cca506f7" status: 200 len: 1315 microversion: 2.1 time: 0.173292 Oct 19 00:49:03.374267 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 749/864] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:49:03 2022] GET /compute/v2.1/servers/813d51c8-3f4b-4caf-ac65-1c18cca506f7 => generated 1315 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:49:04.396174 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-feda3ec8-8bfd-4a0f-a679-878b784ab953 tempest-VolumesNegativeTest-969857559 tempest-VolumesNegativeTest-969857559-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:49:04.396907 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-feda3ec8-8bfd-4a0f-a679-878b784ab953 tempest-VolumesNegativeTest-969857559 tempest-VolumesNegativeTest-969857559-project] [instance: 813d51c8-3f4b-4caf-ac65-1c18cca506f7] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:49:04.403892 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-feda3ec8-8bfd-4a0f-a679-878b784ab953 tempest-VolumesNegativeTest-969857559 tempest-VolumesNegativeTest-969857559-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:49:04.404381 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-feda3ec8-8bfd-4a0f-a679-878b784ab953 tempest-VolumesNegativeTest-969857559 tempest-VolumesNegativeTest-969857559-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:49:04.404815 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-feda3ec8-8bfd-4a0f-a679-878b784ab953 tempest-VolumesNegativeTest-969857559 tempest-VolumesNegativeTest-969857559-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:49:04.405527 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-feda3ec8-8bfd-4a0f-a679-878b784ab953 tempest-VolumesNegativeTest-969857559 tempest-VolumesNegativeTest-969857559-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:49:04.405977 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-feda3ec8-8bfd-4a0f-a679-878b784ab953 tempest-VolumesNegativeTest-969857559 tempest-VolumesNegativeTest-969857559-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:49:04.406408 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-feda3ec8-8bfd-4a0f-a679-878b784ab953 tempest-VolumesNegativeTest-969857559 tempest-VolumesNegativeTest-969857559-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:49:04.593263 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-feda3ec8-8bfd-4a0f-a679-878b784ab953 tempest-VolumesNegativeTest-969857559 tempest-VolumesNegativeTest-969857559-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '796ecf06911540789ec035bc41d52031', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a1efe09c74b46f39c467c6a602abd25', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:49:04.603749 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-feda3ec8-8bfd-4a0f-a679-878b784ab953 tempest-VolumesNegativeTest-969857559 tempest-VolumesNegativeTest-969857559-project] 10.222.0.26 "GET /compute/v2.1/servers/813d51c8-3f4b-4caf-ac65-1c18cca506f7" status: 200 len: 1315 microversion: 2.1 time: 0.211364 Oct 19 00:49:04.604664 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 750/865] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:49:04 2022] GET /compute/v2.1/servers/813d51c8-3f4b-4caf-ac65-1c18cca506f7 => generated 1315 bytes in 213 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 19 00:49:05.636367 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-d54ae88c-140c-41c9-b549-3a17b19d7890 tempest-VolumesNegativeTest-969857559 tempest-VolumesNegativeTest-969857559-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:49:05.637534 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-d54ae88c-140c-41c9-b549-3a17b19d7890 tempest-VolumesNegativeTest-969857559 tempest-VolumesNegativeTest-969857559-project] [instance: 813d51c8-3f4b-4caf-ac65-1c18cca506f7] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:49:05.650310 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d54ae88c-140c-41c9-b549-3a17b19d7890 tempest-VolumesNegativeTest-969857559 tempest-VolumesNegativeTest-969857559-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:49:05.651310 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d54ae88c-140c-41c9-b549-3a17b19d7890 tempest-VolumesNegativeTest-969857559 tempest-VolumesNegativeTest-969857559-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:49:05.652247 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d54ae88c-140c-41c9-b549-3a17b19d7890 tempest-VolumesNegativeTest-969857559 tempest-VolumesNegativeTest-969857559-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:49:05.653554 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d54ae88c-140c-41c9-b549-3a17b19d7890 tempest-VolumesNegativeTest-969857559 tempest-VolumesNegativeTest-969857559-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:49:05.654403 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d54ae88c-140c-41c9-b549-3a17b19d7890 tempest-VolumesNegativeTest-969857559 tempest-VolumesNegativeTest-969857559-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:49:05.655238 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d54ae88c-140c-41c9-b549-3a17b19d7890 tempest-VolumesNegativeTest-969857559 tempest-VolumesNegativeTest-969857559-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:49:05.840368 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-d54ae88c-140c-41c9-b549-3a17b19d7890 tempest-VolumesNegativeTest-969857559 tempest-VolumesNegativeTest-969857559-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '796ecf06911540789ec035bc41d52031', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a1efe09c74b46f39c467c6a602abd25', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:49:05.850600 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-d54ae88c-140c-41c9-b549-3a17b19d7890 tempest-VolumesNegativeTest-969857559 tempest-VolumesNegativeTest-969857559-project] 10.222.0.26 "GET /compute/v2.1/servers/813d51c8-3f4b-4caf-ac65-1c18cca506f7" status: 200 len: 1315 microversion: 2.1 time: 0.222100 Oct 19 00:49:05.851197 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 751/866] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:49:05 2022] GET /compute/v2.1/servers/813d51c8-3f4b-4caf-ac65-1c18cca506f7 => generated 1315 bytes in 225 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:49:06.874575 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-34533f85-d791-440e-be06-39b47dca3e78 tempest-VolumesNegativeTest-969857559 tempest-VolumesNegativeTest-969857559-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:49:06.875903 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-34533f85-d791-440e-be06-39b47dca3e78 tempest-VolumesNegativeTest-969857559 tempest-VolumesNegativeTest-969857559-project] [instance: 813d51c8-3f4b-4caf-ac65-1c18cca506f7] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:49:06.882745 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-34533f85-d791-440e-be06-39b47dca3e78 tempest-VolumesNegativeTest-969857559 tempest-VolumesNegativeTest-969857559-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:49:06.883237 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-34533f85-d791-440e-be06-39b47dca3e78 tempest-VolumesNegativeTest-969857559 tempest-VolumesNegativeTest-969857559-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:49:06.883675 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-34533f85-d791-440e-be06-39b47dca3e78 tempest-VolumesNegativeTest-969857559 tempest-VolumesNegativeTest-969857559-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:49:06.884391 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-34533f85-d791-440e-be06-39b47dca3e78 tempest-VolumesNegativeTest-969857559 tempest-VolumesNegativeTest-969857559-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:49:06.884847 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-34533f85-d791-440e-be06-39b47dca3e78 tempest-VolumesNegativeTest-969857559 tempest-VolumesNegativeTest-969857559-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:49:06.885301 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-34533f85-d791-440e-be06-39b47dca3e78 tempest-VolumesNegativeTest-969857559 tempest-VolumesNegativeTest-969857559-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:49:07.046461 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-34533f85-d791-440e-be06-39b47dca3e78 tempest-VolumesNegativeTest-969857559 tempest-VolumesNegativeTest-969857559-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '796ecf06911540789ec035bc41d52031', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a1efe09c74b46f39c467c6a602abd25', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:49:07.055202 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-34533f85-d791-440e-be06-39b47dca3e78 tempest-VolumesNegativeTest-969857559 tempest-VolumesNegativeTest-969857559-project] 10.222.0.26 "GET /compute/v2.1/servers/813d51c8-3f4b-4caf-ac65-1c18cca506f7" status: 200 len: 1315 microversion: 2.1 time: 0.184540 Oct 19 00:49:07.055783 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 752/867] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:49:06 2022] GET /compute/v2.1/servers/813d51c8-3f4b-4caf-ac65-1c18cca506f7 => generated 1315 bytes in 186 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:49:07.718598 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [req-c725bae8-1498-4817-b00b-c3fe3908e2e0 req-5f75eb25-880a-4fa5-97c2-7a1c93c1c69b service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "813d51c8-3f4b-4caf-ac65-1c18cca506f7", "name": "network-vif-plugged", "status": "completed", "tag": "1c927b91-ee97-413c-b510-50f66cc4fd5c"}]} {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 19 00:49:07.731206 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-c725bae8-1498-4817-b00b-c3fe3908e2e0 req-5f75eb25-880a-4fa5-97c2-7a1c93c1c69b service nova] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:49:07.733314 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-c725bae8-1498-4817-b00b-c3fe3908e2e0 req-5f75eb25-880a-4fa5-97c2-7a1c93c1c69b service nova] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:49:07.733775 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-c725bae8-1498-4817-b00b-c3fe3908e2e0 req-5f75eb25-880a-4fa5-97c2-7a1c93c1c69b service nova] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:49:07.753294 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.compute.server_external_events [req-c725bae8-1498-4817-b00b-c3fe3908e2e0 req-5f75eb25-880a-4fa5-97c2-7a1c93c1c69b service nova] Creating event network-vif-plugged:1c927b91-ee97-413c-b510-50f66cc4fd5c for instance 813d51c8-3f4b-4caf-ac65-1c18cca506f7 on hv-cinder-maste Oct 19 00:49:07.765149 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [req-c725bae8-1498-4817-b00b-c3fe3908e2e0 req-5f75eb25-880a-4fa5-97c2-7a1c93c1c69b service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.055178 Oct 19 00:49:07.765149 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 753/868] 10.222.0.26 () {64 vars in 1317 bytes} [Wed Oct 19 00:49:07 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 19 00:49:08.140378 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-ef32c091-dcf1-44ab-b73a-ed212ac2c81c tempest-VolumesNegativeTest-969857559 tempest-VolumesNegativeTest-969857559-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:49:08.141617 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-ef32c091-dcf1-44ab-b73a-ed212ac2c81c tempest-VolumesNegativeTest-969857559 tempest-VolumesNegativeTest-969857559-project] [instance: 813d51c8-3f4b-4caf-ac65-1c18cca506f7] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:49:08.153534 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ef32c091-dcf1-44ab-b73a-ed212ac2c81c tempest-VolumesNegativeTest-969857559 tempest-VolumesNegativeTest-969857559-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:49:08.154379 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ef32c091-dcf1-44ab-b73a-ed212ac2c81c tempest-VolumesNegativeTest-969857559 tempest-VolumesNegativeTest-969857559-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:49:08.155258 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ef32c091-dcf1-44ab-b73a-ed212ac2c81c tempest-VolumesNegativeTest-969857559 tempest-VolumesNegativeTest-969857559-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:49:08.157101 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ef32c091-dcf1-44ab-b73a-ed212ac2c81c tempest-VolumesNegativeTest-969857559 tempest-VolumesNegativeTest-969857559-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:49:08.158015 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ef32c091-dcf1-44ab-b73a-ed212ac2c81c tempest-VolumesNegativeTest-969857559 tempest-VolumesNegativeTest-969857559-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:49:08.158769 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ef32c091-dcf1-44ab-b73a-ed212ac2c81c tempest-VolumesNegativeTest-969857559 tempest-VolumesNegativeTest-969857559-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:49:08.349629 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-ef32c091-dcf1-44ab-b73a-ed212ac2c81c tempest-VolumesNegativeTest-969857559 tempest-VolumesNegativeTest-969857559-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '796ecf06911540789ec035bc41d52031', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a1efe09c74b46f39c467c6a602abd25', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:49:08.363498 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-ef32c091-dcf1-44ab-b73a-ed212ac2c81c tempest-VolumesNegativeTest-969857559 tempest-VolumesNegativeTest-969857559-project] 10.222.0.26 "GET /compute/v2.1/servers/813d51c8-3f4b-4caf-ac65-1c18cca506f7" status: 200 len: 1315 microversion: 2.1 time: 0.228702 Oct 19 00:49:08.364326 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 754/869] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:49:08 2022] GET /compute/v2.1/servers/813d51c8-3f4b-4caf-ac65-1c18cca506f7 => generated 1315 bytes in 231 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 19 00:49:09.402307 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-3fdf0832-d3ab-4d32-b1ab-ecc0a3eaef70 tempest-VolumesNegativeTest-969857559 tempest-VolumesNegativeTest-969857559-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:49:09.403404 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-3fdf0832-d3ab-4d32-b1ab-ecc0a3eaef70 tempest-VolumesNegativeTest-969857559 tempest-VolumesNegativeTest-969857559-project] [instance: 813d51c8-3f4b-4caf-ac65-1c18cca506f7] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:49:09.408477 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG dbcounter [-] [125641] Writing DB stats nova_cell0:SELECT=2 {{(pid=125641) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 00:49:09.410055 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG dbcounter [-] [125641] Writing DB stats nova_cell0:SELECT=8 {{(pid=125641) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 00:49:09.420080 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-3fdf0832-d3ab-4d32-b1ab-ecc0a3eaef70 tempest-VolumesNegativeTest-969857559 tempest-VolumesNegativeTest-969857559-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:49:09.420589 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-3fdf0832-d3ab-4d32-b1ab-ecc0a3eaef70 tempest-VolumesNegativeTest-969857559 tempest-VolumesNegativeTest-969857559-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:49:09.421026 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-3fdf0832-d3ab-4d32-b1ab-ecc0a3eaef70 tempest-VolumesNegativeTest-969857559 tempest-VolumesNegativeTest-969857559-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:49:09.421747 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-3fdf0832-d3ab-4d32-b1ab-ecc0a3eaef70 tempest-VolumesNegativeTest-969857559 tempest-VolumesNegativeTest-969857559-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:49:09.422397 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-3fdf0832-d3ab-4d32-b1ab-ecc0a3eaef70 tempest-VolumesNegativeTest-969857559 tempest-VolumesNegativeTest-969857559-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:49:09.422867 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-3fdf0832-d3ab-4d32-b1ab-ecc0a3eaef70 tempest-VolumesNegativeTest-969857559 tempest-VolumesNegativeTest-969857559-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:49:09.616011 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-3fdf0832-d3ab-4d32-b1ab-ecc0a3eaef70 tempest-VolumesNegativeTest-969857559 tempest-VolumesNegativeTest-969857559-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '796ecf06911540789ec035bc41d52031', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a1efe09c74b46f39c467c6a602abd25', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:49:09.630143 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-3fdf0832-d3ab-4d32-b1ab-ecc0a3eaef70 tempest-VolumesNegativeTest-969857559 tempest-VolumesNegativeTest-969857559-project] 10.222.0.26 "GET /compute/v2.1/servers/813d51c8-3f4b-4caf-ac65-1c18cca506f7" status: 200 len: 1315 microversion: 2.1 time: 0.232746 Oct 19 00:49:09.631063 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 755/870] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:49:09 2022] GET /compute/v2.1/servers/813d51c8-3f4b-4caf-ac65-1c18cca506f7 => generated 1315 bytes in 235 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:49:10.668877 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-8023e9c3-0af4-4c0a-9a43-d05be1f85d1d tempest-VolumesNegativeTest-969857559 tempest-VolumesNegativeTest-969857559-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:49:10.669987 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-8023e9c3-0af4-4c0a-9a43-d05be1f85d1d tempest-VolumesNegativeTest-969857559 tempest-VolumesNegativeTest-969857559-project] [instance: 813d51c8-3f4b-4caf-ac65-1c18cca506f7] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:49:10.682025 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-8023e9c3-0af4-4c0a-9a43-d05be1f85d1d tempest-VolumesNegativeTest-969857559 tempest-VolumesNegativeTest-969857559-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:49:10.682953 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-8023e9c3-0af4-4c0a-9a43-d05be1f85d1d tempest-VolumesNegativeTest-969857559 tempest-VolumesNegativeTest-969857559-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:49:10.683764 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-8023e9c3-0af4-4c0a-9a43-d05be1f85d1d tempest-VolumesNegativeTest-969857559 tempest-VolumesNegativeTest-969857559-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:49:10.685003 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-8023e9c3-0af4-4c0a-9a43-d05be1f85d1d tempest-VolumesNegativeTest-969857559 tempest-VolumesNegativeTest-969857559-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:49:10.688633 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-8023e9c3-0af4-4c0a-9a43-d05be1f85d1d tempest-VolumesNegativeTest-969857559 tempest-VolumesNegativeTest-969857559-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:49:10.689146 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-8023e9c3-0af4-4c0a-9a43-d05be1f85d1d tempest-VolumesNegativeTest-969857559 tempest-VolumesNegativeTest-969857559-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:49:10.889420 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-8023e9c3-0af4-4c0a-9a43-d05be1f85d1d tempest-VolumesNegativeTest-969857559 tempest-VolumesNegativeTest-969857559-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '796ecf06911540789ec035bc41d52031', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a1efe09c74b46f39c467c6a602abd25', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:49:10.904960 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-8023e9c3-0af4-4c0a-9a43-d05be1f85d1d tempest-VolumesNegativeTest-969857559 tempest-VolumesNegativeTest-969857559-project] 10.222.0.26 "GET /compute/v2.1/servers/813d51c8-3f4b-4caf-ac65-1c18cca506f7" status: 200 len: 1465 microversion: 2.1 time: 0.243258 Oct 19 00:49:10.905843 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 756/871] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:49:10 2022] GET /compute/v2.1/servers/813d51c8-3f4b-4caf-ac65-1c18cca506f7 => generated 1465 bytes in 246 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:49:10.984099 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-00e563d7-ad31-42e4-a5bb-e18133675a53 tempest-VolumesNegativeTest-969857559 tempest-VolumesNegativeTest-969857559-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:49:10.985495 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-00e563d7-ad31-42e4-a5bb-e18133675a53 tempest-VolumesNegativeTest-969857559 tempest-VolumesNegativeTest-969857559-project] [instance: 813d51c8-3f4b-4caf-ac65-1c18cca506f7] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:49:10.993822 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-00e563d7-ad31-42e4-a5bb-e18133675a53 tempest-VolumesNegativeTest-969857559 tempest-VolumesNegativeTest-969857559-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:49:10.994439 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-00e563d7-ad31-42e4-a5bb-e18133675a53 tempest-VolumesNegativeTest-969857559 tempest-VolumesNegativeTest-969857559-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:49:10.995281 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-00e563d7-ad31-42e4-a5bb-e18133675a53 tempest-VolumesNegativeTest-969857559 tempest-VolumesNegativeTest-969857559-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:49:10.996049 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-00e563d7-ad31-42e4-a5bb-e18133675a53 tempest-VolumesNegativeTest-969857559 tempest-VolumesNegativeTest-969857559-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:49:10.996760 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-00e563d7-ad31-42e4-a5bb-e18133675a53 tempest-VolumesNegativeTest-969857559 tempest-VolumesNegativeTest-969857559-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:49:10.997438 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-00e563d7-ad31-42e4-a5bb-e18133675a53 tempest-VolumesNegativeTest-969857559 tempest-VolumesNegativeTest-969857559-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:49:11.033500 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-00e563d7-ad31-42e4-a5bb-e18133675a53 tempest-VolumesNegativeTest-969857559 tempest-VolumesNegativeTest-969857559-project] [instance: 813d51c8-3f4b-4caf-ac65-1c18cca506f7] Going to try to terminate instance {{(pid=125641) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 19 00:49:11.169843 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-00e563d7-ad31-42e4-a5bb-e18133675a53 tempest-VolumesNegativeTest-969857559 tempest-VolumesNegativeTest-969857559-project] 10.222.0.26 "DELETE /compute/v2.1/servers/813d51c8-3f4b-4caf-ac65-1c18cca506f7" status: 204 len: 0 microversion: 2.1 time: 0.191930 Oct 19 00:49:11.170411 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 757/872] 10.222.0.26 () {58 vars in 1239 bytes} [Wed Oct 19 00:49:10 2022] DELETE /compute/v2.1/servers/813d51c8-3f4b-4caf-ac65-1c18cca506f7 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 9 headers in 363 bytes (2 switches on core 0) Oct 19 00:49:11.183812 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-8c33b872-bf8b-45fa-bd33-a78adbdd4778 tempest-VolumesNegativeTest-969857559 tempest-VolumesNegativeTest-969857559-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:49:11.184640 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-8c33b872-bf8b-45fa-bd33-a78adbdd4778 tempest-VolumesNegativeTest-969857559 tempest-VolumesNegativeTest-969857559-project] [instance: 813d51c8-3f4b-4caf-ac65-1c18cca506f7] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:49:11.193701 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-8c33b872-bf8b-45fa-bd33-a78adbdd4778 tempest-VolumesNegativeTest-969857559 tempest-VolumesNegativeTest-969857559-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:49:11.194423 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-8c33b872-bf8b-45fa-bd33-a78adbdd4778 tempest-VolumesNegativeTest-969857559 tempest-VolumesNegativeTest-969857559-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:49:11.195166 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-8c33b872-bf8b-45fa-bd33-a78adbdd4778 tempest-VolumesNegativeTest-969857559 tempest-VolumesNegativeTest-969857559-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:49:11.196039 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-8c33b872-bf8b-45fa-bd33-a78adbdd4778 tempest-VolumesNegativeTest-969857559 tempest-VolumesNegativeTest-969857559-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:49:11.196581 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-8c33b872-bf8b-45fa-bd33-a78adbdd4778 tempest-VolumesNegativeTest-969857559 tempest-VolumesNegativeTest-969857559-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:49:11.197144 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-8c33b872-bf8b-45fa-bd33-a78adbdd4778 tempest-VolumesNegativeTest-969857559 tempest-VolumesNegativeTest-969857559-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:49:11.365301 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-8c33b872-bf8b-45fa-bd33-a78adbdd4778 tempest-VolumesNegativeTest-969857559 tempest-VolumesNegativeTest-969857559-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '796ecf06911540789ec035bc41d52031', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a1efe09c74b46f39c467c6a602abd25', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:49:11.376284 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-8c33b872-bf8b-45fa-bd33-a78adbdd4778 tempest-VolumesNegativeTest-969857559 tempest-VolumesNegativeTest-969857559-project] 10.222.0.26 "GET /compute/v2.1/servers/813d51c8-3f4b-4caf-ac65-1c18cca506f7" status: 200 len: 1471 microversion: 2.1 time: 0.198516 Oct 19 00:49:11.376997 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 758/873] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:49:11 2022] GET /compute/v2.1/servers/813d51c8-3f4b-4caf-ac65-1c18cca506f7 => generated 1471 bytes in 201 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:49:12.400871 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-ff0ea7bf-4ecc-4972-8ee6-d77e600eec29 tempest-VolumesNegativeTest-969857559 tempest-VolumesNegativeTest-969857559-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:49:12.401620 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-ff0ea7bf-4ecc-4972-8ee6-d77e600eec29 tempest-VolumesNegativeTest-969857559 tempest-VolumesNegativeTest-969857559-project] [instance: 813d51c8-3f4b-4caf-ac65-1c18cca506f7] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:49:12.409104 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ff0ea7bf-4ecc-4972-8ee6-d77e600eec29 tempest-VolumesNegativeTest-969857559 tempest-VolumesNegativeTest-969857559-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:49:12.409615 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ff0ea7bf-4ecc-4972-8ee6-d77e600eec29 tempest-VolumesNegativeTest-969857559 tempest-VolumesNegativeTest-969857559-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:49:12.410082 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ff0ea7bf-4ecc-4972-8ee6-d77e600eec29 tempest-VolumesNegativeTest-969857559 tempest-VolumesNegativeTest-969857559-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:49:12.410823 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ff0ea7bf-4ecc-4972-8ee6-d77e600eec29 tempest-VolumesNegativeTest-969857559 tempest-VolumesNegativeTest-969857559-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:49:12.411283 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ff0ea7bf-4ecc-4972-8ee6-d77e600eec29 tempest-VolumesNegativeTest-969857559 tempest-VolumesNegativeTest-969857559-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:49:12.411709 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ff0ea7bf-4ecc-4972-8ee6-d77e600eec29 tempest-VolumesNegativeTest-969857559 tempest-VolumesNegativeTest-969857559-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:49:12.597357 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-ff0ea7bf-4ecc-4972-8ee6-d77e600eec29 tempest-VolumesNegativeTest-969857559 tempest-VolumesNegativeTest-969857559-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '796ecf06911540789ec035bc41d52031', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a1efe09c74b46f39c467c6a602abd25', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:49:12.605814 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-ff0ea7bf-4ecc-4972-8ee6-d77e600eec29 tempest-VolumesNegativeTest-969857559 tempest-VolumesNegativeTest-969857559-project] 10.222.0.26 "GET /compute/v2.1/servers/813d51c8-3f4b-4caf-ac65-1c18cca506f7" status: 200 len: 1471 microversion: 2.1 time: 0.209086 Oct 19 00:49:12.606478 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 759/874] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:49:12 2022] GET /compute/v2.1/servers/813d51c8-3f4b-4caf-ac65-1c18cca506f7 => generated 1471 bytes in 210 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 19 00:49:13.403860 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [req-68f10fda-9bce-4c1a-9817-fe141f78feb9 req-ab08ed6d-07a0-4b9c-8188-3fe3aa4a1061 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "813d51c8-3f4b-4caf-ac65-1c18cca506f7", "name": "network-vif-deleted", "tag": "1c927b91-ee97-413c-b510-50f66cc4fd5c"}]} {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 19 00:49:13.420077 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-68f10fda-9bce-4c1a-9817-fe141f78feb9 req-ab08ed6d-07a0-4b9c-8188-3fe3aa4a1061 service nova] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:49:13.420475 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-68f10fda-9bce-4c1a-9817-fe141f78feb9 req-ab08ed6d-07a0-4b9c-8188-3fe3aa4a1061 service nova] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:49:13.420999 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-68f10fda-9bce-4c1a-9817-fe141f78feb9 req-ab08ed6d-07a0-4b9c-8188-3fe3aa4a1061 service nova] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:49:13.452501 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.compute.server_external_events [req-68f10fda-9bce-4c1a-9817-fe141f78feb9 req-ab08ed6d-07a0-4b9c-8188-3fe3aa4a1061 service nova] Creating event network-vif-deleted:1c927b91-ee97-413c-b510-50f66cc4fd5c for instance 813d51c8-3f4b-4caf-ac65-1c18cca506f7 on hv-cinder-maste Oct 19 00:49:13.462636 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [req-68f10fda-9bce-4c1a-9817-fe141f78feb9 req-ab08ed6d-07a0-4b9c-8188-3fe3aa4a1061 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.062190 Oct 19 00:49:13.463389 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 760/875] 10.222.0.26 () {64 vars in 1317 bytes} [Wed Oct 19 00:49:13 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 64 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 19 00:49:13.638173 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-9eb9f6a1-01b8-4c12-8499-375f1234c27e tempest-VolumesNegativeTest-969857559 tempest-VolumesNegativeTest-969857559-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:49:13.639144 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-9eb9f6a1-01b8-4c12-8499-375f1234c27e tempest-VolumesNegativeTest-969857559 tempest-VolumesNegativeTest-969857559-project] [instance: 813d51c8-3f4b-4caf-ac65-1c18cca506f7] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:49:13.647164 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-9eb9f6a1-01b8-4c12-8499-375f1234c27e tempest-VolumesNegativeTest-969857559 tempest-VolumesNegativeTest-969857559-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:49:13.648001 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-9eb9f6a1-01b8-4c12-8499-375f1234c27e tempest-VolumesNegativeTest-969857559 tempest-VolumesNegativeTest-969857559-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:49:13.648556 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-9eb9f6a1-01b8-4c12-8499-375f1234c27e tempest-VolumesNegativeTest-969857559 tempest-VolumesNegativeTest-969857559-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:49:13.649414 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-9eb9f6a1-01b8-4c12-8499-375f1234c27e tempest-VolumesNegativeTest-969857559 tempest-VolumesNegativeTest-969857559-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:49:13.650043 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-9eb9f6a1-01b8-4c12-8499-375f1234c27e tempest-VolumesNegativeTest-969857559 tempest-VolumesNegativeTest-969857559-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:49:13.650672 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-9eb9f6a1-01b8-4c12-8499-375f1234c27e tempest-VolumesNegativeTest-969857559 tempest-VolumesNegativeTest-969857559-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:49:13.681932 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG dbcounter [-] [125641] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=125641) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 00:49:13.711988 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-9eb9f6a1-01b8-4c12-8499-375f1234c27e tempest-VolumesNegativeTest-969857559 tempest-VolumesNegativeTest-969857559-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:49:13.712914 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-9eb9f6a1-01b8-4c12-8499-375f1234c27e tempest-VolumesNegativeTest-969857559 tempest-VolumesNegativeTest-969857559-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:49:13.713769 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-9eb9f6a1-01b8-4c12-8499-375f1234c27e tempest-VolumesNegativeTest-969857559 tempest-VolumesNegativeTest-969857559-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:49:13.714659 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.objects.instance [None req-9eb9f6a1-01b8-4c12-8499-375f1234c27e tempest-VolumesNegativeTest-969857559 tempest-VolumesNegativeTest-969857559-project] Lazy-loading 'fault' on Instance uuid 813d51c8-3f4b-4caf-ac65-1c18cca506f7 {{(pid=125641) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 19 00:49:13.776816 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-9eb9f6a1-01b8-4c12-8499-375f1234c27e tempest-VolumesNegativeTest-969857559 tempest-VolumesNegativeTest-969857559-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '796ecf06911540789ec035bc41d52031', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a1efe09c74b46f39c467c6a602abd25', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:49:13.788591 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-9eb9f6a1-01b8-4c12-8499-375f1234c27e tempest-VolumesNegativeTest-969857559 tempest-VolumesNegativeTest-969857559-project] 10.222.0.26 "GET /compute/v2.1/servers/813d51c8-3f4b-4caf-ac65-1c18cca506f7" status: 200 len: 1305 microversion: 2.1 time: 0.155217 Oct 19 00:49:13.789259 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 761/876] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:49:13 2022] GET /compute/v2.1/servers/813d51c8-3f4b-4caf-ac65-1c18cca506f7 => generated 1305 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 19 00:49:14.817055 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-f68b15dd-457c-4675-9743-3f8c54bc085f tempest-VolumesNegativeTest-969857559 tempest-VolumesNegativeTest-969857559-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:49:14.817839 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-f68b15dd-457c-4675-9743-3f8c54bc085f tempest-VolumesNegativeTest-969857559 tempest-VolumesNegativeTest-969857559-project] [instance: 813d51c8-3f4b-4caf-ac65-1c18cca506f7] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:49:14.828463 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-f68b15dd-457c-4675-9743-3f8c54bc085f tempest-VolumesNegativeTest-969857559 tempest-VolumesNegativeTest-969857559-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:49:14.829383 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-f68b15dd-457c-4675-9743-3f8c54bc085f tempest-VolumesNegativeTest-969857559 tempest-VolumesNegativeTest-969857559-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:49:14.830260 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-f68b15dd-457c-4675-9743-3f8c54bc085f tempest-VolumesNegativeTest-969857559 tempest-VolumesNegativeTest-969857559-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:49:14.831871 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-f68b15dd-457c-4675-9743-3f8c54bc085f tempest-VolumesNegativeTest-969857559 tempest-VolumesNegativeTest-969857559-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:49:14.832748 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-f68b15dd-457c-4675-9743-3f8c54bc085f tempest-VolumesNegativeTest-969857559 tempest-VolumesNegativeTest-969857559-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:49:14.833664 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-f68b15dd-457c-4675-9743-3f8c54bc085f tempest-VolumesNegativeTest-969857559 tempest-VolumesNegativeTest-969857559-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:49:14.871292 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.wsgi [None req-f68b15dd-457c-4675-9743-3f8c54bc085f tempest-VolumesNegativeTest-969857559 tempest-VolumesNegativeTest-969857559-project] HTTP exception thrown: Instance 813d51c8-3f4b-4caf-ac65-1c18cca506f7 could not be found. Oct 19 00:49:14.874967 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-f68b15dd-457c-4675-9743-3f8c54bc085f tempest-VolumesNegativeTest-969857559 tempest-VolumesNegativeTest-969857559-project] Request method failure captured: Oct 19 00:49:14.874967 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: request: GET /compute/v2.1/servers/813d51c8-3f4b-4caf-ac65-1c18cca506f7 HTTP/1.1 Oct 19 00:49:14.874967 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Accept: application/json Oct 19 00:49:14.874967 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Accept-Encoding: identity Oct 19 00:49:14.874967 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Connection: close Oct 19 00:49:14.874967 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Content-Length: 0 Oct 19 00:49:14.874967 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Content-Type: application/json Oct 19 00:49:14.874967 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Host: 10.222.0.26 Oct 19 00:49:14.874967 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Openstack-System-Scope: None Oct 19 00:49:14.874967 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: User-Agent: python-urllib3/1.26.12 Oct 19 00:49:14.874967 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Auth-Token: gAAAAABjT0l4zx5szvE1XdBlXSGItEo_gMQflfDfxdL_PLMTuazACxPx5uQjRpgU-Y53vZL-_6KiQuJ988enPEC2hhSF_eYOE6q8byU7BSJaSWh9IdVpe_Wl22OcNqe10E7kNmvzib6tdeS1LPJdV4ziCb2J2ClkEwkZUet99s9apMC0iazlP24 Oct 19 00:49:14.874967 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Domain-Id: None Oct 19 00:49:14.874967 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Domain-Name: None Oct 19 00:49:14.874967 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Identity-Status: Confirmed Oct 19 00:49:14.874967 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Is-Admin-Project: True Oct 19 00:49:14.874967 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Project-Domain-Id: default Oct 19 00:49:14.874967 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Project-Domain-Name: Default Oct 19 00:49:14.874967 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Project-Id: 2a1efe09c74b46f39c467c6a602abd25 Oct 19 00:49:14.874967 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Project-Name: tempest-VolumesNegativeTest-969857559 Oct 19 00:49:14.874967 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Role: member,reader Oct 19 00:49:14.874967 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Roles: member,reader Oct 19 00:49:14.878864 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/2a1efe09c74b46f39c467c6a602abd25"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/2a1efe09c74b46f39c467c6a602abd25"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/2a1efe09c74b46f39c467c6a602abd25"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_2a1efe09c74b46f39c467c6a602abd25"}]}] Oct 19 00:49:14.878864 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Tenant: tempest-VolumesNegativeTest-969857559 Oct 19 00:49:14.878864 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Tenant-Id: 2a1efe09c74b46f39c467c6a602abd25 Oct 19 00:49:14.878864 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Tenant-Name: tempest-VolumesNegativeTest-969857559 Oct 19 00:49:14.878864 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User: tempest-VolumesNegativeTest-969857559-project Oct 19 00:49:14.878864 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User-Domain-Id: default Oct 19 00:49:14.878864 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User-Domain-Name: Default Oct 19 00:49:14.878864 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User-Id: 796ecf06911540789ec035bc41d52031 Oct 19 00:49:14.878864 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User-Name: tempest-VolumesNegativeTest-969857559-project Oct 19 00:49:14.878864 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: method: > Oct 19 00:49:14.878864 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: exception: Instance 813d51c8-3f4b-4caf-ac65-1c18cca506f7 could not be found. Oct 19 00:49:14.878864 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: action_args: {'id': '813d51c8-3f4b-4caf-ac65-1c18cca506f7'} Oct 19 00:49:14.886390 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 19 00:49:14.886390 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 19 00:49:14.886390 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 19 00:49:14.886390 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 19 00:49:14.886390 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 19 00:49:14.886390 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 19 00:49:14.886390 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 19 00:49:14.886390 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 19 00:49:14.886390 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 19 00:49:14.886390 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 19 00:49:14.886390 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 813d51c8-3f4b-4caf-ac65-1c18cca506f7 could not be found. Oct 19 00:49:14.886390 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:49:14.886390 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 19 00:49:14.886390 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:49:14.886390 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 19 00:49:14.886390 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 19 00:49:14.886390 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 19 00:49:14.886390 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 19 00:49:14.886390 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 19 00:49:14.886390 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 19 00:49:14.889994 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 19 00:49:14.889994 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 19 00:49:14.889994 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 19 00:49:14.889994 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 19 00:49:14.889994 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 19 00:49:14.889994 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 19 00:49:14.889994 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 19 00:49:14.889994 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 813d51c8-3f4b-4caf-ac65-1c18cca506f7 could not be found. Oct 19 00:49:14.889994 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:49:14.889994 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 19 00:49:14.889994 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:49:14.889994 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 19 00:49:14.889994 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 19 00:49:14.889994 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 19 00:49:14.889994 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 19 00:49:14.889994 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 19 00:49:14.889994 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 813d51c8-3f4b-4caf-ac65-1c18cca506f7 could not be found. Oct 19 00:49:14.889994 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:49:14.892980 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-f68b15dd-457c-4675-9743-3f8c54bc085f tempest-VolumesNegativeTest-969857559 tempest-VolumesNegativeTest-969857559-project] Returning 404 to user: Instance 813d51c8-3f4b-4caf-ac65-1c18cca506f7 could not be found. {{(pid=125641) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 19 00:49:14.892980 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-f68b15dd-457c-4675-9743-3f8c54bc085f tempest-VolumesNegativeTest-969857559 tempest-VolumesNegativeTest-969857559-project] 10.222.0.26 "GET /compute/v2.1/servers/813d51c8-3f4b-4caf-ac65-1c18cca506f7" status: 404 len: 111 microversion: 2.1 time: 0.067081 Oct 19 00:49:14.892980 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 762/877] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:49:14 2022] GET /compute/v2.1/servers/813d51c8-3f4b-4caf-ac65-1c18cca506f7 => generated 111 bytes in 69 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Oct 19 00:49:22.862557 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-647c918f-2069-451f-8de8-a2b2fa4ba8fd tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:49:22.866310 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-647c918f-2069-451f-8de8-a2b2fa4ba8fd tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] [instance: b38d4219-8912-4fd9-9d02-7848016f7954] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:49:22.878837 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-647c918f-2069-451f-8de8-a2b2fa4ba8fd tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:49:22.879796 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-647c918f-2069-451f-8de8-a2b2fa4ba8fd tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:49:22.880614 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-647c918f-2069-451f-8de8-a2b2fa4ba8fd tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:49:22.881844 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-647c918f-2069-451f-8de8-a2b2fa4ba8fd tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:49:22.882734 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-647c918f-2069-451f-8de8-a2b2fa4ba8fd tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:49:22.883495 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-647c918f-2069-451f-8de8-a2b2fa4ba8fd tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:49:22.939785 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-647c918f-2069-451f-8de8-a2b2fa4ba8fd tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] 10.222.0.26 "GET /compute/v2.1/servers/b38d4219-8912-4fd9-9d02-7848016f7954/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.084813 Oct 19 00:49:22.940740 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 763/878] 10.222.0.26 () {58 vars in 1302 bytes} [Wed Oct 19 00:49:22 2022] GET /compute/v2.1/servers/b38d4219-8912-4fd9-9d02-7848016f7954/os-volume_attachments => generated 25 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Oct 19 00:49:23.015914 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-93d3ff8a-5a78-4673-ba00-312e77f1b0b8 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:49:23.016532 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-93d3ff8a-5a78-4673-ba00-312e77f1b0b8 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] [instance: b38d4219-8912-4fd9-9d02-7848016f7954] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:49:23.023219 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-93d3ff8a-5a78-4673-ba00-312e77f1b0b8 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:49:23.023723 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-93d3ff8a-5a78-4673-ba00-312e77f1b0b8 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:49:23.024166 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-93d3ff8a-5a78-4673-ba00-312e77f1b0b8 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:49:23.024893 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-93d3ff8a-5a78-4673-ba00-312e77f1b0b8 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:49:23.025359 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-93d3ff8a-5a78-4673-ba00-312e77f1b0b8 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:49:23.025787 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-93d3ff8a-5a78-4673-ba00-312e77f1b0b8 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:49:23.055206 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-93d3ff8a-5a78-4673-ba00-312e77f1b0b8 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] [instance: b38d4219-8912-4fd9-9d02-7848016f7954] Going to try to terminate instance {{(pid=125641) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 19 00:49:23.150076 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-93d3ff8a-5a78-4673-ba00-312e77f1b0b8 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] 10.222.0.26 "DELETE /compute/v2.1/servers/b38d4219-8912-4fd9-9d02-7848016f7954" status: 204 len: 0 microversion: 2.1 time: 0.137562 Oct 19 00:49:23.156267 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 764/879] 10.222.0.26 () {58 vars in 1239 bytes} [Wed Oct 19 00:49:23 2022] DELETE /compute/v2.1/servers/b38d4219-8912-4fd9-9d02-7848016f7954 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 9 headers in 363 bytes (2 switches on core 0) Oct 19 00:49:23.166673 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-064f7ffd-55c7-4c33-9cb0-b2cebb2b2e86 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:49:23.167880 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-064f7ffd-55c7-4c33-9cb0-b2cebb2b2e86 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] [instance: b38d4219-8912-4fd9-9d02-7848016f7954] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:49:23.178993 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-064f7ffd-55c7-4c33-9cb0-b2cebb2b2e86 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:49:23.179875 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-064f7ffd-55c7-4c33-9cb0-b2cebb2b2e86 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:49:23.180841 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-064f7ffd-55c7-4c33-9cb0-b2cebb2b2e86 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:49:23.182147 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-064f7ffd-55c7-4c33-9cb0-b2cebb2b2e86 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:49:23.183082 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-064f7ffd-55c7-4c33-9cb0-b2cebb2b2e86 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:49:23.183943 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-064f7ffd-55c7-4c33-9cb0-b2cebb2b2e86 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:49:23.341606 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-064f7ffd-55c7-4c33-9cb0-b2cebb2b2e86 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a48ed2d9ded741ca91afc1fbbd9c608f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '82d7723ef95b4c3faeac9835eeb233a5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:49:23.350626 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-064f7ffd-55c7-4c33-9cb0-b2cebb2b2e86 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] 10.222.0.26 "GET /compute/v2.1/servers/b38d4219-8912-4fd9-9d02-7848016f7954" status: 200 len: 1675 microversion: 2.1 time: 0.191543 Oct 19 00:49:23.351297 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 765/880] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:49:23 2022] GET /compute/v2.1/servers/b38d4219-8912-4fd9-9d02-7848016f7954 => generated 1675 bytes in 194 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:49:24.378936 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-00ad7383-4d34-48e3-9951-eb274bff7893 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:49:24.380024 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-00ad7383-4d34-48e3-9951-eb274bff7893 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] [instance: b38d4219-8912-4fd9-9d02-7848016f7954] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:49:24.387271 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-00ad7383-4d34-48e3-9951-eb274bff7893 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:49:24.387753 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-00ad7383-4d34-48e3-9951-eb274bff7893 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:49:24.388195 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-00ad7383-4d34-48e3-9951-eb274bff7893 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:49:24.388918 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-00ad7383-4d34-48e3-9951-eb274bff7893 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:49:24.389381 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-00ad7383-4d34-48e3-9951-eb274bff7893 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:49:24.389804 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-00ad7383-4d34-48e3-9951-eb274bff7893 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:49:24.539374 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-00ad7383-4d34-48e3-9951-eb274bff7893 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a48ed2d9ded741ca91afc1fbbd9c608f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '82d7723ef95b4c3faeac9835eeb233a5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:49:24.549423 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-00ad7383-4d34-48e3-9951-eb274bff7893 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] 10.222.0.26 "GET /compute/v2.1/servers/b38d4219-8912-4fd9-9d02-7848016f7954" status: 200 len: 1675 microversion: 2.1 time: 0.176836 Oct 19 00:49:24.550643 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 766/881] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:49:24 2022] GET /compute/v2.1/servers/b38d4219-8912-4fd9-9d02-7848016f7954 => generated 1675 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 19 00:49:25.578425 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-e297fcdf-ff6c-451b-b565-1503098e660f tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:49:25.579721 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-e297fcdf-ff6c-451b-b565-1503098e660f tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] [instance: b38d4219-8912-4fd9-9d02-7848016f7954] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:49:25.586659 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-e297fcdf-ff6c-451b-b565-1503098e660f tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:49:25.587215 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-e297fcdf-ff6c-451b-b565-1503098e660f tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:49:25.587696 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-e297fcdf-ff6c-451b-b565-1503098e660f tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:49:25.588585 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-e297fcdf-ff6c-451b-b565-1503098e660f tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:49:25.589113 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-e297fcdf-ff6c-451b-b565-1503098e660f tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:49:25.589910 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-e297fcdf-ff6c-451b-b565-1503098e660f tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:49:25.762342 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-e297fcdf-ff6c-451b-b565-1503098e660f tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a48ed2d9ded741ca91afc1fbbd9c608f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '82d7723ef95b4c3faeac9835eeb233a5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:49:25.772347 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-e297fcdf-ff6c-451b-b565-1503098e660f tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] 10.222.0.26 "GET /compute/v2.1/servers/b38d4219-8912-4fd9-9d02-7848016f7954" status: 200 len: 1675 microversion: 2.1 time: 0.198167 Oct 19 00:49:25.772934 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 767/882] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:49:25 2022] GET /compute/v2.1/servers/b38d4219-8912-4fd9-9d02-7848016f7954 => generated 1675 bytes in 200 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 19 00:49:26.797636 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-ba04ed04-175b-4c90-b739-91b1936ad6dd tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:49:26.798409 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-ba04ed04-175b-4c90-b739-91b1936ad6dd tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] [instance: b38d4219-8912-4fd9-9d02-7848016f7954] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:49:26.804970 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ba04ed04-175b-4c90-b739-91b1936ad6dd tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:49:26.805477 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ba04ed04-175b-4c90-b739-91b1936ad6dd tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:49:26.805917 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ba04ed04-175b-4c90-b739-91b1936ad6dd tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:49:26.806765 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ba04ed04-175b-4c90-b739-91b1936ad6dd tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:49:26.807149 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ba04ed04-175b-4c90-b739-91b1936ad6dd tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:49:26.807582 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ba04ed04-175b-4c90-b739-91b1936ad6dd tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:49:26.979992 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-ba04ed04-175b-4c90-b739-91b1936ad6dd tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a48ed2d9ded741ca91afc1fbbd9c608f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '82d7723ef95b4c3faeac9835eeb233a5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:49:26.988438 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-ba04ed04-175b-4c90-b739-91b1936ad6dd tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] 10.222.0.26 "GET /compute/v2.1/servers/b38d4219-8912-4fd9-9d02-7848016f7954" status: 200 len: 1675 microversion: 2.1 time: 0.194907 Oct 19 00:49:26.989122 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 768/883] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:49:26 2022] GET /compute/v2.1/servers/b38d4219-8912-4fd9-9d02-7848016f7954 => generated 1675 bytes in 196 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 19 00:49:28.018169 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-c66673cd-a220-4c5a-a309-1a613b55369d tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:49:28.019309 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-c66673cd-a220-4c5a-a309-1a613b55369d tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] [instance: b38d4219-8912-4fd9-9d02-7848016f7954] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:49:28.023508 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG dbcounter [-] [125641] Writing DB stats nova_api:SELECT=78,nova_api:INSERT=3,nova_api:UPDATE=2 {{(pid=125641) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 00:49:28.034289 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-c66673cd-a220-4c5a-a309-1a613b55369d tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:49:28.035196 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-c66673cd-a220-4c5a-a309-1a613b55369d tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:49:28.036025 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-c66673cd-a220-4c5a-a309-1a613b55369d tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:49:28.037450 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-c66673cd-a220-4c5a-a309-1a613b55369d tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:49:28.038272 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-c66673cd-a220-4c5a-a309-1a613b55369d tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:49:28.039124 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-c66673cd-a220-4c5a-a309-1a613b55369d tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:49:28.225097 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-c66673cd-a220-4c5a-a309-1a613b55369d tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a48ed2d9ded741ca91afc1fbbd9c608f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '82d7723ef95b4c3faeac9835eeb233a5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:49:28.239231 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-c66673cd-a220-4c5a-a309-1a613b55369d tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] 10.222.0.26 "GET /compute/v2.1/servers/b38d4219-8912-4fd9-9d02-7848016f7954" status: 200 len: 1675 microversion: 2.1 time: 0.228802 Oct 19 00:49:28.240271 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 769/884] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:49:28 2022] GET /compute/v2.1/servers/b38d4219-8912-4fd9-9d02-7848016f7954 => generated 1675 bytes in 231 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:49:29.278844 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-0031efbe-596c-4fb1-a96e-4b3544f5cde3 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:49:29.279551 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-0031efbe-596c-4fb1-a96e-4b3544f5cde3 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] [instance: b38d4219-8912-4fd9-9d02-7848016f7954] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:49:29.287687 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-0031efbe-596c-4fb1-a96e-4b3544f5cde3 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:49:29.288176 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-0031efbe-596c-4fb1-a96e-4b3544f5cde3 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:49:29.288612 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-0031efbe-596c-4fb1-a96e-4b3544f5cde3 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:49:29.289332 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-0031efbe-596c-4fb1-a96e-4b3544f5cde3 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:49:29.289789 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-0031efbe-596c-4fb1-a96e-4b3544f5cde3 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:49:29.290420 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-0031efbe-596c-4fb1-a96e-4b3544f5cde3 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:49:29.445759 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-0031efbe-596c-4fb1-a96e-4b3544f5cde3 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a48ed2d9ded741ca91afc1fbbd9c608f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '82d7723ef95b4c3faeac9835eeb233a5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:49:29.460822 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-0031efbe-596c-4fb1-a96e-4b3544f5cde3 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] 10.222.0.26 "GET /compute/v2.1/servers/b38d4219-8912-4fd9-9d02-7848016f7954" status: 200 len: 1675 microversion: 2.1 time: 0.185707 Oct 19 00:49:29.461880 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 770/885] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:49:29 2022] GET /compute/v2.1/servers/b38d4219-8912-4fd9-9d02-7848016f7954 => generated 1675 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 19 00:49:30.485845 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-a30b2d61-ed7e-44d4-b172-54a485698622 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:49:30.486630 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-a30b2d61-ed7e-44d4-b172-54a485698622 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] [instance: b38d4219-8912-4fd9-9d02-7848016f7954] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:49:30.493277 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-a30b2d61-ed7e-44d4-b172-54a485698622 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:49:30.493766 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-a30b2d61-ed7e-44d4-b172-54a485698622 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:49:30.494194 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-a30b2d61-ed7e-44d4-b172-54a485698622 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:49:30.494943 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-a30b2d61-ed7e-44d4-b172-54a485698622 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:49:30.495388 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-a30b2d61-ed7e-44d4-b172-54a485698622 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:49:30.495806 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-a30b2d61-ed7e-44d4-b172-54a485698622 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:49:30.642353 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-a30b2d61-ed7e-44d4-b172-54a485698622 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a48ed2d9ded741ca91afc1fbbd9c608f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '82d7723ef95b4c3faeac9835eeb233a5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:49:30.650663 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-a30b2d61-ed7e-44d4-b172-54a485698622 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] 10.222.0.26 "GET /compute/v2.1/servers/b38d4219-8912-4fd9-9d02-7848016f7954" status: 200 len: 1675 microversion: 2.1 time: 0.169377 Oct 19 00:49:30.651248 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 771/886] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:49:30 2022] GET /compute/v2.1/servers/b38d4219-8912-4fd9-9d02-7848016f7954 => generated 1675 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 19 00:49:31.677141 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-f5cec7f3-909e-4c1a-891a-94c78f0179c5 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:49:31.677839 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-f5cec7f3-909e-4c1a-891a-94c78f0179c5 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] [instance: b38d4219-8912-4fd9-9d02-7848016f7954] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:49:31.684707 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-f5cec7f3-909e-4c1a-891a-94c78f0179c5 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:49:31.685194 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-f5cec7f3-909e-4c1a-891a-94c78f0179c5 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:49:31.685632 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-f5cec7f3-909e-4c1a-891a-94c78f0179c5 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:49:31.686346 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-f5cec7f3-909e-4c1a-891a-94c78f0179c5 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:49:31.686842 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-f5cec7f3-909e-4c1a-891a-94c78f0179c5 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:49:31.687450 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-f5cec7f3-909e-4c1a-891a-94c78f0179c5 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:49:31.837854 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-f5cec7f3-909e-4c1a-891a-94c78f0179c5 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a48ed2d9ded741ca91afc1fbbd9c608f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '82d7723ef95b4c3faeac9835eeb233a5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:49:31.848065 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-f5cec7f3-909e-4c1a-891a-94c78f0179c5 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] 10.222.0.26 "GET /compute/v2.1/servers/b38d4219-8912-4fd9-9d02-7848016f7954" status: 200 len: 1675 microversion: 2.1 time: 0.175471 Oct 19 00:49:31.848935 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 772/887] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:49:31 2022] GET /compute/v2.1/servers/b38d4219-8912-4fd9-9d02-7848016f7954 => generated 1675 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 19 00:49:32.876034 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-6b6c0d55-52e9-484b-b11c-41f83bceb2fc tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:49:32.876828 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-6b6c0d55-52e9-484b-b11c-41f83bceb2fc tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] [instance: b38d4219-8912-4fd9-9d02-7848016f7954] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:49:32.883810 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-6b6c0d55-52e9-484b-b11c-41f83bceb2fc tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:49:32.884294 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-6b6c0d55-52e9-484b-b11c-41f83bceb2fc tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:49:32.884728 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-6b6c0d55-52e9-484b-b11c-41f83bceb2fc tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:49:32.885432 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-6b6c0d55-52e9-484b-b11c-41f83bceb2fc tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:49:32.885883 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-6b6c0d55-52e9-484b-b11c-41f83bceb2fc tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:49:32.886305 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-6b6c0d55-52e9-484b-b11c-41f83bceb2fc tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:49:32.957377 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-6b6c0d55-52e9-484b-b11c-41f83bceb2fc tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a48ed2d9ded741ca91afc1fbbd9c608f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '82d7723ef95b4c3faeac9835eeb233a5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:49:32.966367 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-6b6c0d55-52e9-484b-b11c-41f83bceb2fc tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] 10.222.0.26 "GET /compute/v2.1/servers/b38d4219-8912-4fd9-9d02-7848016f7954" status: 200 len: 1607 microversion: 2.1 time: 0.096224 Oct 19 00:49:32.967000 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 773/888] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:49:32 2022] GET /compute/v2.1/servers/b38d4219-8912-4fd9-9d02-7848016f7954 => generated 1607 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 19 00:49:33.529460 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [req-96863a8d-5d4f-4607-bec2-9cae0f2aede9 req-860541d1-4608-43e6-a838-20cf4a3a812a service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "b38d4219-8912-4fd9-9d02-7848016f7954", "name": "network-vif-deleted", "tag": "2b936cf8-a250-46ff-af9b-c10ccce59df7"}]} {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 19 00:49:33.568801 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-96863a8d-5d4f-4607-bec2-9cae0f2aede9 req-860541d1-4608-43e6-a838-20cf4a3a812a service nova] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:49:33.569690 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-96863a8d-5d4f-4607-bec2-9cae0f2aede9 req-860541d1-4608-43e6-a838-20cf4a3a812a service nova] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:49:33.570470 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-96863a8d-5d4f-4607-bec2-9cae0f2aede9 req-860541d1-4608-43e6-a838-20cf4a3a812a service nova] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:49:33.593467 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.compute.server_external_events [req-96863a8d-5d4f-4607-bec2-9cae0f2aede9 req-860541d1-4608-43e6-a838-20cf4a3a812a service nova] Creating event network-vif-deleted:2b936cf8-a250-46ff-af9b-c10ccce59df7 for instance b38d4219-8912-4fd9-9d02-7848016f7954 on hv-cinder-maste Oct 19 00:49:33.603738 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [req-96863a8d-5d4f-4607-bec2-9cae0f2aede9 req-860541d1-4608-43e6-a838-20cf4a3a812a service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.091739 Oct 19 00:49:33.605033 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 774/889] 10.222.0.26 () {64 vars in 1317 bytes} [Wed Oct 19 00:49:33 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 19 00:49:33.997071 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-80062520-5fd6-4b6d-856c-1fd77bf5a241 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:49:33.998280 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-80062520-5fd6-4b6d-856c-1fd77bf5a241 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] [instance: b38d4219-8912-4fd9-9d02-7848016f7954] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:49:34.007087 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-80062520-5fd6-4b6d-856c-1fd77bf5a241 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:49:34.007587 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-80062520-5fd6-4b6d-856c-1fd77bf5a241 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:49:34.008029 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-80062520-5fd6-4b6d-856c-1fd77bf5a241 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:49:34.008756 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-80062520-5fd6-4b6d-856c-1fd77bf5a241 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:49:34.009215 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-80062520-5fd6-4b6d-856c-1fd77bf5a241 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:49:34.009838 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-80062520-5fd6-4b6d-856c-1fd77bf5a241 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:49:34.089556 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-80062520-5fd6-4b6d-856c-1fd77bf5a241 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a48ed2d9ded741ca91afc1fbbd9c608f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '82d7723ef95b4c3faeac9835eeb233a5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:49:34.099200 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-80062520-5fd6-4b6d-856c-1fd77bf5a241 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] 10.222.0.26 "GET /compute/v2.1/servers/b38d4219-8912-4fd9-9d02-7848016f7954" status: 200 len: 1607 microversion: 2.1 time: 0.109401 Oct 19 00:49:34.099809 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 775/890] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:49:33 2022] GET /compute/v2.1/servers/b38d4219-8912-4fd9-9d02-7848016f7954 => generated 1607 bytes in 111 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 19 00:49:35.126761 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-00cd007c-ec5b-4823-947f-62b55a9e9010 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:49:35.127956 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-00cd007c-ec5b-4823-947f-62b55a9e9010 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] [instance: b38d4219-8912-4fd9-9d02-7848016f7954] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:49:35.135502 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-00cd007c-ec5b-4823-947f-62b55a9e9010 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:49:35.136201 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-00cd007c-ec5b-4823-947f-62b55a9e9010 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:49:35.137120 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-00cd007c-ec5b-4823-947f-62b55a9e9010 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:49:35.138056 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-00cd007c-ec5b-4823-947f-62b55a9e9010 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:49:35.138678 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-00cd007c-ec5b-4823-947f-62b55a9e9010 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:49:35.139227 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-00cd007c-ec5b-4823-947f-62b55a9e9010 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:49:35.235974 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-00cd007c-ec5b-4823-947f-62b55a9e9010 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a48ed2d9ded741ca91afc1fbbd9c608f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '82d7723ef95b4c3faeac9835eeb233a5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:49:35.244766 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-00cd007c-ec5b-4823-947f-62b55a9e9010 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] 10.222.0.26 "GET /compute/v2.1/servers/b38d4219-8912-4fd9-9d02-7848016f7954" status: 200 len: 1323 microversion: 2.1 time: 0.123597 Oct 19 00:49:35.245392 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 776/891] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:49:35 2022] GET /compute/v2.1/servers/b38d4219-8912-4fd9-9d02-7848016f7954 => generated 1323 bytes in 125 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 19 00:49:36.281166 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-af568ef8-6dc4-475d-acea-1039ed78a747 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:49:36.281972 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-af568ef8-6dc4-475d-acea-1039ed78a747 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] [instance: b38d4219-8912-4fd9-9d02-7848016f7954] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:49:36.291150 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-af568ef8-6dc4-475d-acea-1039ed78a747 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:49:36.291915 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-af568ef8-6dc4-475d-acea-1039ed78a747 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:49:36.292668 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-af568ef8-6dc4-475d-acea-1039ed78a747 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:49:36.294017 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-af568ef8-6dc4-475d-acea-1039ed78a747 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:49:36.294838 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-af568ef8-6dc4-475d-acea-1039ed78a747 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:49:36.295679 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-af568ef8-6dc4-475d-acea-1039ed78a747 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:49:36.318245 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.wsgi [None req-af568ef8-6dc4-475d-acea-1039ed78a747 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] HTTP exception thrown: Instance b38d4219-8912-4fd9-9d02-7848016f7954 could not be found. Oct 19 00:49:36.320324 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-af568ef8-6dc4-475d-acea-1039ed78a747 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Request method failure captured: Oct 19 00:49:36.320324 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: request: GET /compute/v2.1/servers/b38d4219-8912-4fd9-9d02-7848016f7954 HTTP/1.1 Oct 19 00:49:36.320324 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Accept: application/json Oct 19 00:49:36.320324 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Accept-Encoding: identity Oct 19 00:49:36.320324 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Connection: close Oct 19 00:49:36.320324 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Content-Length: 0 Oct 19 00:49:36.320324 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Content-Type: application/json Oct 19 00:49:36.320324 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Host: 10.222.0.26 Oct 19 00:49:36.320324 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Openstack-System-Scope: None Oct 19 00:49:36.320324 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: User-Agent: python-urllib3/1.26.12 Oct 19 00:49:36.320324 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Auth-Token: gAAAAABjT0jftXzQfZyc8CRHQV4W_gddvzuI97tl8zjpIOHAUUDNwJcgCLYc0G45lhVAYuRkTnFQ_q2IREDQvezXcz0pdRx04PeX9oCthNL6COthZ_7lYmlxWElgdQkmAYCIVWlggqLn29lN1bm77huNlmbpHl2KvOIdUnCRzvDMG4IjBMuh1Jc Oct 19 00:49:36.320324 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Domain-Id: None Oct 19 00:49:36.320324 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Domain-Name: None Oct 19 00:49:36.320324 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Identity-Status: Confirmed Oct 19 00:49:36.320324 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Is-Admin-Project: True Oct 19 00:49:36.320324 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Project-Domain-Id: default Oct 19 00:49:36.320324 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Project-Domain-Name: Default Oct 19 00:49:36.320324 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Project-Id: 82d7723ef95b4c3faeac9835eeb233a5 Oct 19 00:49:36.320324 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Project-Name: tempest-AttachVolumeTestJSON-35550062 Oct 19 00:49:36.320324 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Role: member,reader Oct 19 00:49:36.320324 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Roles: member,reader Oct 19 00:49:36.323479 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/82d7723ef95b4c3faeac9835eeb233a5"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/82d7723ef95b4c3faeac9835eeb233a5"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/82d7723ef95b4c3faeac9835eeb233a5"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_82d7723ef95b4c3faeac9835eeb233a5"}]}] Oct 19 00:49:36.323479 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Tenant: tempest-AttachVolumeTestJSON-35550062 Oct 19 00:49:36.323479 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Tenant-Id: 82d7723ef95b4c3faeac9835eeb233a5 Oct 19 00:49:36.323479 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Tenant-Name: tempest-AttachVolumeTestJSON-35550062 Oct 19 00:49:36.323479 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User: tempest-AttachVolumeTestJSON-35550062-project Oct 19 00:49:36.323479 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User-Domain-Id: default Oct 19 00:49:36.323479 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User-Domain-Name: Default Oct 19 00:49:36.323479 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User-Id: a48ed2d9ded741ca91afc1fbbd9c608f Oct 19 00:49:36.323479 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User-Name: tempest-AttachVolumeTestJSON-35550062-project Oct 19 00:49:36.323479 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: method: > Oct 19 00:49:36.323479 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: exception: Instance b38d4219-8912-4fd9-9d02-7848016f7954 could not be found. Oct 19 00:49:36.323479 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: action_args: {'id': 'b38d4219-8912-4fd9-9d02-7848016f7954'} Oct 19 00:49:36.325526 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 19 00:49:36.325526 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 19 00:49:36.325526 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 19 00:49:36.325526 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 19 00:49:36.325526 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 19 00:49:36.325526 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 19 00:49:36.325526 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 19 00:49:36.325526 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 19 00:49:36.325526 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 19 00:49:36.325526 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 19 00:49:36.325526 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance b38d4219-8912-4fd9-9d02-7848016f7954 could not be found. Oct 19 00:49:36.325526 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:49:36.325526 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 19 00:49:36.325526 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:49:36.325526 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 19 00:49:36.325526 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 19 00:49:36.325526 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 19 00:49:36.325526 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 19 00:49:36.325526 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 19 00:49:36.325526 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 19 00:49:36.328189 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 19 00:49:36.328189 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 19 00:49:36.328189 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 19 00:49:36.328189 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 19 00:49:36.328189 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 19 00:49:36.328189 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 19 00:49:36.328189 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 19 00:49:36.328189 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance b38d4219-8912-4fd9-9d02-7848016f7954 could not be found. Oct 19 00:49:36.328189 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:49:36.328189 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 19 00:49:36.328189 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:49:36.328189 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 19 00:49:36.328189 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 19 00:49:36.328189 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 19 00:49:36.328189 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 19 00:49:36.328189 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 19 00:49:36.328189 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance b38d4219-8912-4fd9-9d02-7848016f7954 could not be found. Oct 19 00:49:36.328189 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:49:36.330446 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-af568ef8-6dc4-475d-acea-1039ed78a747 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Returning 404 to user: Instance b38d4219-8912-4fd9-9d02-7848016f7954 could not be found. {{(pid=125641) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 19 00:49:36.330446 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-af568ef8-6dc4-475d-acea-1039ed78a747 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] 10.222.0.26 "GET /compute/v2.1/servers/b38d4219-8912-4fd9-9d02-7848016f7954" status: 404 len: 111 microversion: 2.1 time: 0.048768 Oct 19 00:49:36.330446 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 777/892] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:49:36 2022] GET /compute/v2.1/servers/b38d4219-8912-4fd9-9d02-7848016f7954 => generated 111 bytes in 51 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Oct 19 00:49:36.336694 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-7da657ce-8238-420f-ba95-a0dbb2460747 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Calling method '.version_select at 0x7f71cb61c430>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:49:36.370830 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-7da657ce-8238-420f-ba95-a0dbb2460747 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] 10.222.0.26 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-1137457954" status: 202 len: 0 microversion: 2.1 time: 0.038313 Oct 19 00:49:36.371425 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 778/893] 10.222.0.26 () {58 vars in 1221 bytes} [Wed Oct 19 00:49:36 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-1137457954 => generated 0 bytes in 41 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 19 00:49:37.486926 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-15f26d70-d3e7-4bea-a88e-bfbbc71d68bf tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Action: 'create', calling method: .version_select at 0x7f71cb5cf430>, body: {"keypair": {"name": "tempest-keypair-2113229266", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBKwn/BxfkC8QuAQXmX5UQ5+5Y8tFroo1Uuw2llEVwG2iXl4tMNyx5zv2vfl3FkyiJpQVhFYQrMgSwIuGqrXgeSnBnpvoN9dMz3/d8YZtxfg6Zz7i+oozxt/ye3r2Qo17Gg=="}} {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 19 00:49:37.518719 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.quota [None req-15f26d70-d3e7-4bea-a88e-bfbbc71d68bf tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Getting quotas for project 82d7723ef95b4c3faeac9835eeb233a5. Resources: {'key_pairs'} {{(pid=125641) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 19 00:49:37.526543 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.quota [None req-15f26d70-d3e7-4bea-a88e-bfbbc71d68bf tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Getting quotas for user a48ed2d9ded741ca91afc1fbbd9c608f and project 82d7723ef95b4c3faeac9835eeb233a5. Resources: {'key_pairs'} {{(pid=125641) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 19 00:49:37.566884 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-15f26d70-d3e7-4bea-a88e-bfbbc71d68bf tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] 10.222.0.26 "POST /compute/v2.1/os-keypairs" status: 200 len: 386 microversion: 2.1 time: 0.089611 Oct 19 00:49:37.567500 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 779/894] 10.222.0.26 () {60 vars in 1159 bytes} [Wed Oct 19 00:49:37 2022] POST /compute/v2.1/os-keypairs => generated 386 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 19 00:49:38.383553 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-45503509-f60c-40d7-8209-f7a7f7db11c0 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestVolumeBootPattern-server-249980678", "imageRef": "", "flavorRef": "452", "key_name": "tempest-TestVolumeBootPattern-1204843348", "security_groups": [{"name": "tempest-secgroup-smoke-1056832705"}], "block_device_mapping_v2": [{"uuid": "c8f49bda-b744-4d60-9485-ce9dbbf0a973", "source_type": "volume", "destination_type": "volume", "boot_index": 0, "delete_on_termination": false}], "networks": [{"uuid": "cfe47441-dd27-4b7c-958a-cc2e5c8243ac"}]}} {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 19 00:49:38.638351 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.network.neutron [None req-45503509-f60c-40d7-8209-f7a7f7db11c0 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] validate_networks() for [('cfe47441-dd27-4b7c-958a-cc2e5c8243ac', None, None, None, None, None)] {{(pid=125641) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 19 00:49:39.520755 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.virt.hardware [None req-45503509-f60c-40d7-8209-f7a7f7db11c0 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Flavor limits 0:0:0 {{(pid=125641) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 19 00:49:39.521760 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.virt.hardware [None req-45503509-f60c-40d7-8209-f7a7f7db11c0 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Image limits 0:0:0 {{(pid=125641) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 19 00:49:39.522954 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.virt.hardware [None req-45503509-f60c-40d7-8209-f7a7f7db11c0 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Flavor pref 0:0:0 {{(pid=125641) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 19 00:49:39.523708 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.virt.hardware [None req-45503509-f60c-40d7-8209-f7a7f7db11c0 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Image pref 0:0:0 {{(pid=125641) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 19 00:49:39.524380 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.virt.hardware [None req-45503509-f60c-40d7-8209-f7a7f7db11c0 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=125641) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 19 00:49:39.543038 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-45503509-f60c-40d7-8209-f7a7f7db11c0 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:49:39.543038 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-45503509-f60c-40d7-8209-f7a7f7db11c0 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:49:39.543573 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-45503509-f60c-40d7-8209-f7a7f7db11c0 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:49:39.545050 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-45503509-f60c-40d7-8209-f7a7f7db11c0 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:49:39.545948 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-45503509-f60c-40d7-8209-f7a7f7db11c0 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:49:39.546987 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-45503509-f60c-40d7-8209-f7a7f7db11c0 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:49:39.564575 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG dbcounter [-] [125641] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=125641) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 00:49:39.582616 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.quota [None req-45503509-f60c-40d7-8209-f7a7f7db11c0 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Getting quotas for project 8684504b0eff415b9063cebae21ca667. Resources: {'cores', 'ram', 'instances'} {{(pid=125641) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 19 00:49:39.589653 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.quota [None req-45503509-f60c-40d7-8209-f7a7f7db11c0 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Getting quotas for user f738e67d8c2340028da1254b60d7810e and project 8684504b0eff415b9063cebae21ca667. Resources: {'cores', 'ram', 'instances'} {{(pid=125641) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 19 00:49:39.682059 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-45503509-f60c-40d7-8209-f7a7f7db11c0 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Going to run 1 instances... {{(pid=125641) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 19 00:49:39.686337 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-45503509-f60c-40d7-8209-f7a7f7db11c0 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] [instance: ae2afe94-f67b-419f-bd5f-aefc5fbce221] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=False,deleted=,deleted_at=,destination_type='volume',device_name=None,device_type=None,disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id=None,instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='volume',tag=None,updated_at=,uuid=,volume_id='c8f49bda-b744-4d60-9485-ce9dbbf0a973',volume_size=None,volume_type=None)] {{(pid=125641) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 19 00:49:39.855114 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-45503509-f60c-40d7-8209-f7a7f7db11c0 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 202 len: 406 microversion: 2.1 time: 1.479464 Oct 19 00:49:39.855782 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 780/895] 10.222.0.26 () {60 vars in 1147 bytes} [Wed Oct 19 00:49:38 2022] POST /compute/v2.1/servers => generated 406 bytes in 1482 msecs (HTTP/1.1 202) 10 headers in 451 bytes (2 switches on core 0) Oct 19 00:49:39.861784 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-bd2541a6-3120-432e-ab68-fc400651bc8c tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeTestJSON-server-1487003848", "imageRef": "b387b4f8-f75e-4a68-b49c-a015bf61c46b", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "68404256-b068-4f6b-bb54-21abe9c04e36"}], "security_groups": [{"name": "tempest-securitygroup--1482965379"}], "key_name": "tempest-keypair-2113229266", "user_data": "}} {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 19 00:49:40.034562 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.network.neutron [None req-bd2541a6-3120-432e-ab68-fc400651bc8c tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] validate_networks() for [('68404256-b068-4f6b-bb54-21abe9c04e36', None, None, None, None, None)] {{(pid=125641) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 19 00:49:40.663504 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.virt.hardware [None req-bd2541a6-3120-432e-ab68-fc400651bc8c tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Flavor limits 0:0:0 {{(pid=125641) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 19 00:49:40.664174 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.virt.hardware [None req-bd2541a6-3120-432e-ab68-fc400651bc8c tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Image limits 0:0:0 {{(pid=125641) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 19 00:49:40.664955 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.virt.hardware [None req-bd2541a6-3120-432e-ab68-fc400651bc8c tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Flavor pref 0:0:0 {{(pid=125641) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 19 00:49:40.665514 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.virt.hardware [None req-bd2541a6-3120-432e-ab68-fc400651bc8c tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Image pref 0:0:0 {{(pid=125641) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 19 00:49:40.666046 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.virt.hardware [None req-bd2541a6-3120-432e-ab68-fc400651bc8c tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=125641) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 19 00:49:40.676695 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-bd2541a6-3120-432e-ab68-fc400651bc8c tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:49:40.677189 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-bd2541a6-3120-432e-ab68-fc400651bc8c tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:49:40.677630 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-bd2541a6-3120-432e-ab68-fc400651bc8c tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:49:40.678410 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-bd2541a6-3120-432e-ab68-fc400651bc8c tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:49:40.678900 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-bd2541a6-3120-432e-ab68-fc400651bc8c tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:49:40.679327 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-bd2541a6-3120-432e-ab68-fc400651bc8c tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:49:40.701259 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.quota [None req-bd2541a6-3120-432e-ab68-fc400651bc8c tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Getting quotas for project 82d7723ef95b4c3faeac9835eeb233a5. Resources: {'cores', 'ram', 'instances'} {{(pid=125641) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 19 00:49:40.707298 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.quota [None req-bd2541a6-3120-432e-ab68-fc400651bc8c tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Getting quotas for user a48ed2d9ded741ca91afc1fbbd9c608f and project 82d7723ef95b4c3faeac9835eeb233a5. Resources: {'cores', 'ram', 'instances'} {{(pid=125641) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 19 00:49:40.720921 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-bd2541a6-3120-432e-ab68-fc400651bc8c tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Going to run 1 instances... {{(pid=125641) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 19 00:49:40.723179 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-bd2541a6-3120-432e-ab68-fc400651bc8c tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] [instance: d49a602c-229d-48e6-8b26-c67f73ed8f2f] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='b387b4f8-f75e-4a68-b49c-a015bf61c46b',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=125641) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 19 00:49:40.783565 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-bd2541a6-3120-432e-ab68-fc400651bc8c tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 202 len: 402 microversion: 2.1 time: 0.926537 Oct 19 00:49:40.784642 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 781/896] 10.222.0.26 () {60 vars in 1147 bytes} [Wed Oct 19 00:49:39 2022] POST /compute/v2.1/servers => generated 402 bytes in 928 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Oct 19 00:49:40.793919 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-d7a9ebc2-fa74-4145-a46a-77a73abb10f7 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:49:40.795515 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-d7a9ebc2-fa74-4145-a46a-77a73abb10f7 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] [instance: ae2afe94-f67b-419f-bd5f-aefc5fbce221] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:49:40.808578 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d7a9ebc2-fa74-4145-a46a-77a73abb10f7 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:49:40.809937 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d7a9ebc2-fa74-4145-a46a-77a73abb10f7 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:49:40.811013 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d7a9ebc2-fa74-4145-a46a-77a73abb10f7 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:49:40.813160 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d7a9ebc2-fa74-4145-a46a-77a73abb10f7 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:49:40.814017 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d7a9ebc2-fa74-4145-a46a-77a73abb10f7 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:49:40.814819 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d7a9ebc2-fa74-4145-a46a-77a73abb10f7 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:49:40.940689 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-d7a9ebc2-fa74-4145-a46a-77a73abb10f7 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f738e67d8c2340028da1254b60d7810e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8684504b0eff415b9063cebae21ca667', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:49:40.952292 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-d7a9ebc2-fa74-4145-a46a-77a73abb10f7 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] 10.222.0.26 "GET /compute/v2.1/servers/ae2afe94-f67b-419f-bd5f-aefc5fbce221" status: 200 len: 1192 microversion: 2.1 time: 0.165874 Oct 19 00:49:40.953526 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 782/897] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:49:40 2022] GET /compute/v2.1/servers/ae2afe94-f67b-419f-bd5f-aefc5fbce221 => generated 1192 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:49:40.962640 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-7217403c-d7b0-4736-87f6-970901c140a1 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:49:40.964468 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-7217403c-d7b0-4736-87f6-970901c140a1 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] [instance: d49a602c-229d-48e6-8b26-c67f73ed8f2f] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:49:41.054042 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-7217403c-d7b0-4736-87f6-970901c140a1 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a48ed2d9ded741ca91afc1fbbd9c608f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '82d7723ef95b4c3faeac9835eeb233a5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:49:41.062686 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-7217403c-d7b0-4736-87f6-970901c140a1 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] 10.222.0.26 "GET /compute/v2.1/servers/d49a602c-229d-48e6-8b26-c67f73ed8f2f" status: 200 len: 1238 microversion: 2.1 time: 0.107015 Oct 19 00:49:41.063518 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 783/898] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:49:40 2022] GET /compute/v2.1/servers/d49a602c-229d-48e6-8b26-c67f73ed8f2f => generated 1238 bytes in 109 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:49:41.994072 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-d940cc11-60e6-4357-8b13-e5c9de8308e5 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:49:41.996567 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-d940cc11-60e6-4357-8b13-e5c9de8308e5 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] [instance: ae2afe94-f67b-419f-bd5f-aefc5fbce221] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:49:42.011232 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d940cc11-60e6-4357-8b13-e5c9de8308e5 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:49:42.012112 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d940cc11-60e6-4357-8b13-e5c9de8308e5 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:49:42.013316 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d940cc11-60e6-4357-8b13-e5c9de8308e5 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:49:42.015851 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d940cc11-60e6-4357-8b13-e5c9de8308e5 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:49:42.017038 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d940cc11-60e6-4357-8b13-e5c9de8308e5 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:49:42.018452 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d940cc11-60e6-4357-8b13-e5c9de8308e5 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:49:42.157506 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-d940cc11-60e6-4357-8b13-e5c9de8308e5 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f738e67d8c2340028da1254b60d7810e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8684504b0eff415b9063cebae21ca667', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:49:42.169837 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-d940cc11-60e6-4357-8b13-e5c9de8308e5 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] 10.222.0.26 "GET /compute/v2.1/servers/ae2afe94-f67b-419f-bd5f-aefc5fbce221" status: 200 len: 1210 microversion: 2.1 time: 0.184133 Oct 19 00:49:42.174774 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 784/899] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:49:41 2022] GET /compute/v2.1/servers/ae2afe94-f67b-419f-bd5f-aefc5fbce221 => generated 1210 bytes in 190 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:49:42.179959 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-030ce16d-75f3-48cf-91dd-36f045eeae33 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:49:42.180867 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-030ce16d-75f3-48cf-91dd-36f045eeae33 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] [instance: d49a602c-229d-48e6-8b26-c67f73ed8f2f] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:49:42.205661 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-030ce16d-75f3-48cf-91dd-36f045eeae33 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:49:42.206849 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-030ce16d-75f3-48cf-91dd-36f045eeae33 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:49:42.207897 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-030ce16d-75f3-48cf-91dd-36f045eeae33 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:49:42.209451 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-030ce16d-75f3-48cf-91dd-36f045eeae33 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:49:42.210653 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-030ce16d-75f3-48cf-91dd-36f045eeae33 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:49:42.211697 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-030ce16d-75f3-48cf-91dd-36f045eeae33 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:49:42.325323 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-030ce16d-75f3-48cf-91dd-36f045eeae33 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a48ed2d9ded741ca91afc1fbbd9c608f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '82d7723ef95b4c3faeac9835eeb233a5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:49:42.336760 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-030ce16d-75f3-48cf-91dd-36f045eeae33 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] 10.222.0.26 "GET /compute/v2.1/servers/d49a602c-229d-48e6-8b26-c67f73ed8f2f" status: 200 len: 1290 microversion: 2.1 time: 0.160583 Oct 19 00:49:42.346872 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 785/900] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:49:42 2022] GET /compute/v2.1/servers/d49a602c-229d-48e6-8b26-c67f73ed8f2f => generated 1290 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:49:43.207739 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-91988c58-eeae-4027-b8d9-faa11d7b48a5 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:49:43.209987 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-91988c58-eeae-4027-b8d9-faa11d7b48a5 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] [instance: ae2afe94-f67b-419f-bd5f-aefc5fbce221] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:49:43.224812 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-91988c58-eeae-4027-b8d9-faa11d7b48a5 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:49:43.228434 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-91988c58-eeae-4027-b8d9-faa11d7b48a5 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:49:43.228434 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-91988c58-eeae-4027-b8d9-faa11d7b48a5 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:49:43.228434 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-91988c58-eeae-4027-b8d9-faa11d7b48a5 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:49:43.230597 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-91988c58-eeae-4027-b8d9-faa11d7b48a5 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:49:43.232091 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-91988c58-eeae-4027-b8d9-faa11d7b48a5 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:49:43.482576 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-91988c58-eeae-4027-b8d9-faa11d7b48a5 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f738e67d8c2340028da1254b60d7810e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8684504b0eff415b9063cebae21ca667', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:49:43.503394 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-91988c58-eeae-4027-b8d9-faa11d7b48a5 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] 10.222.0.26 "GET /compute/v2.1/servers/ae2afe94-f67b-419f-bd5f-aefc5fbce221" status: 200 len: 1266 microversion: 2.1 time: 0.302382 Oct 19 00:49:43.504511 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 786/901] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:49:43 2022] GET /compute/v2.1/servers/ae2afe94-f67b-419f-bd5f-aefc5fbce221 => generated 1266 bytes in 305 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 19 00:49:43.515878 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-01741f57-b9d9-45da-8af9-cd8983b087c9 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:49:43.515878 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-01741f57-b9d9-45da-8af9-cd8983b087c9 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] [instance: d49a602c-229d-48e6-8b26-c67f73ed8f2f] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:49:43.533753 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-01741f57-b9d9-45da-8af9-cd8983b087c9 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:49:43.534739 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-01741f57-b9d9-45da-8af9-cd8983b087c9 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:49:43.535527 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-01741f57-b9d9-45da-8af9-cd8983b087c9 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:49:43.536992 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-01741f57-b9d9-45da-8af9-cd8983b087c9 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:49:43.537822 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-01741f57-b9d9-45da-8af9-cd8983b087c9 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:49:43.539410 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-01741f57-b9d9-45da-8af9-cd8983b087c9 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:49:43.647884 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-01741f57-b9d9-45da-8af9-cd8983b087c9 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a48ed2d9ded741ca91afc1fbbd9c608f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '82d7723ef95b4c3faeac9835eeb233a5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:49:43.660999 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-01741f57-b9d9-45da-8af9-cd8983b087c9 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] 10.222.0.26 "GET /compute/v2.1/servers/d49a602c-229d-48e6-8b26-c67f73ed8f2f" status: 200 len: 1296 microversion: 2.1 time: 0.155102 Oct 19 00:49:43.661606 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 787/902] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:49:43 2022] GET /compute/v2.1/servers/d49a602c-229d-48e6-8b26-c67f73ed8f2f => generated 1296 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:49:44.548598 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-d2ce83f9-635b-45b4-b70e-43d4b9d563a5 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:49:44.551644 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-d2ce83f9-635b-45b4-b70e-43d4b9d563a5 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] [instance: ae2afe94-f67b-419f-bd5f-aefc5fbce221] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:49:44.568649 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d2ce83f9-635b-45b4-b70e-43d4b9d563a5 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:49:44.570012 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d2ce83f9-635b-45b4-b70e-43d4b9d563a5 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:49:44.571457 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d2ce83f9-635b-45b4-b70e-43d4b9d563a5 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:49:44.573372 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d2ce83f9-635b-45b4-b70e-43d4b9d563a5 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:49:44.574353 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d2ce83f9-635b-45b4-b70e-43d4b9d563a5 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:49:44.575212 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d2ce83f9-635b-45b4-b70e-43d4b9d563a5 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:49:44.957422 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-d2ce83f9-635b-45b4-b70e-43d4b9d563a5 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f738e67d8c2340028da1254b60d7810e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8684504b0eff415b9063cebae21ca667', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:49:44.987556 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-d2ce83f9-635b-45b4-b70e-43d4b9d563a5 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] 10.222.0.26 "GET /compute/v2.1/servers/ae2afe94-f67b-419f-bd5f-aefc5fbce221" status: 200 len: 1266 microversion: 2.1 time: 0.446573 Oct 19 00:49:45.009550 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 788/903] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:49:44 2022] GET /compute/v2.1/servers/ae2afe94-f67b-419f-bd5f-aefc5fbce221 => generated 1266 bytes in 470 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:49:45.032496 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-d0f270e9-9a15-4fe3-b028-cf6e94a6eeab tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:49:45.035671 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-d0f270e9-9a15-4fe3-b028-cf6e94a6eeab tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] [instance: d49a602c-229d-48e6-8b26-c67f73ed8f2f] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:49:45.047441 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d0f270e9-9a15-4fe3-b028-cf6e94a6eeab tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:49:45.048715 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d0f270e9-9a15-4fe3-b028-cf6e94a6eeab tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:49:45.049790 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d0f270e9-9a15-4fe3-b028-cf6e94a6eeab tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:49:45.051423 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d0f270e9-9a15-4fe3-b028-cf6e94a6eeab tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:49:45.052830 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d0f270e9-9a15-4fe3-b028-cf6e94a6eeab tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:49:45.053916 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d0f270e9-9a15-4fe3-b028-cf6e94a6eeab tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:49:45.691223 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-d0f270e9-9a15-4fe3-b028-cf6e94a6eeab tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a48ed2d9ded741ca91afc1fbbd9c608f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '82d7723ef95b4c3faeac9835eeb233a5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:49:45.704320 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-d0f270e9-9a15-4fe3-b028-cf6e94a6eeab tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] 10.222.0.26 "GET /compute/v2.1/servers/d49a602c-229d-48e6-8b26-c67f73ed8f2f" status: 200 len: 1364 microversion: 2.1 time: 0.679382 Oct 19 00:49:45.705362 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 789/904] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:49:45 2022] GET /compute/v2.1/servers/d49a602c-229d-48e6-8b26-c67f73ed8f2f => generated 1364 bytes in 695 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:49:45.711735 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [req-78cea387-eb1a-49cd-954b-6c2b6032ff5d req-7a8ceb91-4ef9-4fda-b96d-b8522a342961 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "ae2afe94-f67b-419f-bd5f-aefc5fbce221", "tag": "c76271cc-707a-4a90-9ae9-135461e8154f"}]} {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 19 00:49:45.730006 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-78cea387-eb1a-49cd-954b-6c2b6032ff5d req-7a8ceb91-4ef9-4fda-b96d-b8522a342961 service nova] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:49:45.730584 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-78cea387-eb1a-49cd-954b-6c2b6032ff5d req-7a8ceb91-4ef9-4fda-b96d-b8522a342961 service nova] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:49:45.731651 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-78cea387-eb1a-49cd-954b-6c2b6032ff5d req-7a8ceb91-4ef9-4fda-b96d-b8522a342961 service nova] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:49:45.750185 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.compute.server_external_events [req-78cea387-eb1a-49cd-954b-6c2b6032ff5d req-7a8ceb91-4ef9-4fda-b96d-b8522a342961 service nova] Creating event network-changed:c76271cc-707a-4a90-9ae9-135461e8154f for instance ae2afe94-f67b-419f-bd5f-aefc5fbce221 on hv-cinder-maste Oct 19 00:49:45.756665 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [req-78cea387-eb1a-49cd-954b-6c2b6032ff5d req-7a8ceb91-4ef9-4fda-b96d-b8522a342961 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.049580 Oct 19 00:49:45.758061 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 790/905] 10.222.0.26 () {64 vars in 1317 bytes} [Wed Oct 19 00:49:45 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 19 00:49:46.031401 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-73eaf125-457e-43e0-8089-947c5e16df56 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:49:46.036655 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-73eaf125-457e-43e0-8089-947c5e16df56 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] [instance: ae2afe94-f67b-419f-bd5f-aefc5fbce221] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:49:46.044914 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-73eaf125-457e-43e0-8089-947c5e16df56 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:49:46.045900 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-73eaf125-457e-43e0-8089-947c5e16df56 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:49:46.046874 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-73eaf125-457e-43e0-8089-947c5e16df56 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:49:46.047932 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-73eaf125-457e-43e0-8089-947c5e16df56 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:49:46.048687 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-73eaf125-457e-43e0-8089-947c5e16df56 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:49:46.049355 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-73eaf125-457e-43e0-8089-947c5e16df56 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:49:46.344969 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-73eaf125-457e-43e0-8089-947c5e16df56 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f738e67d8c2340028da1254b60d7810e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8684504b0eff415b9063cebae21ca667', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:49:46.361409 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-73eaf125-457e-43e0-8089-947c5e16df56 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] 10.222.0.26 "GET /compute/v2.1/servers/ae2afe94-f67b-419f-bd5f-aefc5fbce221" status: 200 len: 1266 microversion: 2.1 time: 0.334784 Oct 19 00:49:46.379369 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 791/906] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:49:46 2022] GET /compute/v2.1/servers/ae2afe94-f67b-419f-bd5f-aefc5fbce221 => generated 1266 bytes in 336 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 19 00:49:46.741747 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-d809759b-91d5-459c-adfc-fb120f4f231d tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:49:46.744684 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-d809759b-91d5-459c-adfc-fb120f4f231d tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] [instance: d49a602c-229d-48e6-8b26-c67f73ed8f2f] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:49:46.765111 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d809759b-91d5-459c-adfc-fb120f4f231d tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:49:46.766177 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d809759b-91d5-459c-adfc-fb120f4f231d tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:49:46.767025 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d809759b-91d5-459c-adfc-fb120f4f231d tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:49:46.768025 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d809759b-91d5-459c-adfc-fb120f4f231d tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:49:46.768631 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d809759b-91d5-459c-adfc-fb120f4f231d tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:49:46.769337 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d809759b-91d5-459c-adfc-fb120f4f231d tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:49:47.130342 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-d809759b-91d5-459c-adfc-fb120f4f231d tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a48ed2d9ded741ca91afc1fbbd9c608f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '82d7723ef95b4c3faeac9835eeb233a5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:49:47.151052 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-d809759b-91d5-459c-adfc-fb120f4f231d tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] 10.222.0.26 "GET /compute/v2.1/servers/d49a602c-229d-48e6-8b26-c67f73ed8f2f" status: 200 len: 1364 microversion: 2.1 time: 0.423958 Oct 19 00:49:47.156498 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 792/907] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:49:46 2022] GET /compute/v2.1/servers/d49a602c-229d-48e6-8b26-c67f73ed8f2f => generated 1364 bytes in 431 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 19 00:49:47.389237 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-9bcdea2d-60b0-47a0-b172-71e67ddaba23 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:49:47.391982 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-9bcdea2d-60b0-47a0-b172-71e67ddaba23 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] [instance: ae2afe94-f67b-419f-bd5f-aefc5fbce221] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:49:47.401826 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-9bcdea2d-60b0-47a0-b172-71e67ddaba23 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:49:47.402777 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-9bcdea2d-60b0-47a0-b172-71e67ddaba23 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:49:47.403446 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-9bcdea2d-60b0-47a0-b172-71e67ddaba23 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:49:47.404481 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-9bcdea2d-60b0-47a0-b172-71e67ddaba23 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:49:47.405259 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-9bcdea2d-60b0-47a0-b172-71e67ddaba23 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:49:47.406268 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-9bcdea2d-60b0-47a0-b172-71e67ddaba23 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:49:47.713808 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-9bcdea2d-60b0-47a0-b172-71e67ddaba23 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f738e67d8c2340028da1254b60d7810e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8684504b0eff415b9063cebae21ca667', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:49:47.725332 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-9bcdea2d-60b0-47a0-b172-71e67ddaba23 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] 10.222.0.26 "GET /compute/v2.1/servers/ae2afe94-f67b-419f-bd5f-aefc5fbce221" status: 200 len: 1266 microversion: 2.1 time: 0.341450 Oct 19 00:49:47.726347 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 793/908] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:49:47 2022] GET /compute/v2.1/servers/ae2afe94-f67b-419f-bd5f-aefc5fbce221 => generated 1266 bytes in 343 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 19 00:49:47.810458 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [req-01114943-18eb-4948-bb52-433f81ab0855 req-42e18e8d-8989-497a-8364-59d7d3a741bb service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "d49a602c-229d-48e6-8b26-c67f73ed8f2f", "tag": "a6830382-e592-4ea8-9d6a-63da173d5d01"}]} {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 19 00:49:47.822222 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-01114943-18eb-4948-bb52-433f81ab0855 req-42e18e8d-8989-497a-8364-59d7d3a741bb service nova] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:49:47.823296 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-01114943-18eb-4948-bb52-433f81ab0855 req-42e18e8d-8989-497a-8364-59d7d3a741bb service nova] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:49:47.824002 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-01114943-18eb-4948-bb52-433f81ab0855 req-42e18e8d-8989-497a-8364-59d7d3a741bb service nova] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:49:47.841311 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.compute.server_external_events [req-01114943-18eb-4948-bb52-433f81ab0855 req-42e18e8d-8989-497a-8364-59d7d3a741bb service nova] Creating event network-changed:a6830382-e592-4ea8-9d6a-63da173d5d01 for instance d49a602c-229d-48e6-8b26-c67f73ed8f2f on hv-cinder-maste Oct 19 00:49:47.848850 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [req-01114943-18eb-4948-bb52-433f81ab0855 req-42e18e8d-8989-497a-8364-59d7d3a741bb service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.041942 Oct 19 00:49:47.850186 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 794/909] 10.222.0.26 () {64 vars in 1317 bytes} [Wed Oct 19 00:49:47 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 19 00:49:48.191766 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-14f872bc-cad6-4c07-8e7b-8eaa594897ee tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:49:48.193711 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-14f872bc-cad6-4c07-8e7b-8eaa594897ee tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] [instance: d49a602c-229d-48e6-8b26-c67f73ed8f2f] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:49:48.204156 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-14f872bc-cad6-4c07-8e7b-8eaa594897ee tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:49:48.204667 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-14f872bc-cad6-4c07-8e7b-8eaa594897ee tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:49:48.205109 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-14f872bc-cad6-4c07-8e7b-8eaa594897ee tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:49:48.206239 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-14f872bc-cad6-4c07-8e7b-8eaa594897ee tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:49:48.206836 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-14f872bc-cad6-4c07-8e7b-8eaa594897ee tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:49:48.207682 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-14f872bc-cad6-4c07-8e7b-8eaa594897ee tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:49:48.600205 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-14f872bc-cad6-4c07-8e7b-8eaa594897ee tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a48ed2d9ded741ca91afc1fbbd9c608f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '82d7723ef95b4c3faeac9835eeb233a5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:49:48.619088 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-14f872bc-cad6-4c07-8e7b-8eaa594897ee tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] 10.222.0.26 "GET /compute/v2.1/servers/d49a602c-229d-48e6-8b26-c67f73ed8f2f" status: 200 len: 1364 microversion: 2.1 time: 0.434355 Oct 19 00:49:48.620705 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 795/910] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:49:48 2022] GET /compute/v2.1/servers/d49a602c-229d-48e6-8b26-c67f73ed8f2f => generated 1364 bytes in 437 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 19 00:49:48.755230 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-f3704d3b-d738-4042-828f-9b7b2f5b488d tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:49:48.756391 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-f3704d3b-d738-4042-828f-9b7b2f5b488d tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] [instance: ae2afe94-f67b-419f-bd5f-aefc5fbce221] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:49:48.765248 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG dbcounter [-] [125641] Writing DB stats nova_api:SELECT=92,nova_api:DELETE=1,nova_api:INSERT=7 {{(pid=125641) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 00:49:48.769000 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-f3704d3b-d738-4042-828f-9b7b2f5b488d tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:49:48.770079 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-f3704d3b-d738-4042-828f-9b7b2f5b488d tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:49:48.771081 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-f3704d3b-d738-4042-828f-9b7b2f5b488d tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:49:48.772970 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-f3704d3b-d738-4042-828f-9b7b2f5b488d tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:49:48.773776 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-f3704d3b-d738-4042-828f-9b7b2f5b488d tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:49:48.774568 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-f3704d3b-d738-4042-828f-9b7b2f5b488d tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:49:49.095885 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-f3704d3b-d738-4042-828f-9b7b2f5b488d tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f738e67d8c2340028da1254b60d7810e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8684504b0eff415b9063cebae21ca667', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:49:49.112347 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-f3704d3b-d738-4042-828f-9b7b2f5b488d tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] 10.222.0.26 "GET /compute/v2.1/servers/ae2afe94-f67b-419f-bd5f-aefc5fbce221" status: 200 len: 1266 microversion: 2.1 time: 0.365153 Oct 19 00:49:49.113474 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 796/911] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:49:48 2022] GET /compute/v2.1/servers/ae2afe94-f67b-419f-bd5f-aefc5fbce221 => generated 1266 bytes in 368 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 19 00:49:49.663418 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-03d91f16-8a26-4f88-9b94-e73a4ad51f9d tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:49:49.664870 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-03d91f16-8a26-4f88-9b94-e73a4ad51f9d tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] [instance: d49a602c-229d-48e6-8b26-c67f73ed8f2f] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:49:49.699161 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-03d91f16-8a26-4f88-9b94-e73a4ad51f9d tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:49:49.700792 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-03d91f16-8a26-4f88-9b94-e73a4ad51f9d tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:49:49.713566 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-03d91f16-8a26-4f88-9b94-e73a4ad51f9d tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:49:49.715827 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-03d91f16-8a26-4f88-9b94-e73a4ad51f9d tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:49:49.716959 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-03d91f16-8a26-4f88-9b94-e73a4ad51f9d tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:49:49.718639 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-03d91f16-8a26-4f88-9b94-e73a4ad51f9d tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:49:50.107849 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-03d91f16-8a26-4f88-9b94-e73a4ad51f9d tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a48ed2d9ded741ca91afc1fbbd9c608f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '82d7723ef95b4c3faeac9835eeb233a5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:49:50.126663 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-03d91f16-8a26-4f88-9b94-e73a4ad51f9d tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] 10.222.0.26 "GET /compute/v2.1/servers/d49a602c-229d-48e6-8b26-c67f73ed8f2f" status: 200 len: 1364 microversion: 2.1 time: 0.473529 Oct 19 00:49:50.157228 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 797/912] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:49:49 2022] GET /compute/v2.1/servers/d49a602c-229d-48e6-8b26-c67f73ed8f2f => generated 1364 bytes in 492 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:49:50.157228 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-201be186-d47b-4d21-aaa9-ca6eac25aa08 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:49:50.157228 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-201be186-d47b-4d21-aaa9-ca6eac25aa08 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] [instance: ae2afe94-f67b-419f-bd5f-aefc5fbce221] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:49:50.162414 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-201be186-d47b-4d21-aaa9-ca6eac25aa08 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:49:50.163198 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-201be186-d47b-4d21-aaa9-ca6eac25aa08 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:49:50.164494 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-201be186-d47b-4d21-aaa9-ca6eac25aa08 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:49:50.165429 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-201be186-d47b-4d21-aaa9-ca6eac25aa08 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:49:50.166110 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-201be186-d47b-4d21-aaa9-ca6eac25aa08 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:49:50.166585 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-201be186-d47b-4d21-aaa9-ca6eac25aa08 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:49:50.421696 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-201be186-d47b-4d21-aaa9-ca6eac25aa08 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f738e67d8c2340028da1254b60d7810e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8684504b0eff415b9063cebae21ca667', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:49:50.436827 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-201be186-d47b-4d21-aaa9-ca6eac25aa08 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] 10.222.0.26 "GET /compute/v2.1/servers/ae2afe94-f67b-419f-bd5f-aefc5fbce221" status: 200 len: 1266 microversion: 2.1 time: 0.289394 Oct 19 00:49:50.438573 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 798/913] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:49:50 2022] GET /compute/v2.1/servers/ae2afe94-f67b-419f-bd5f-aefc5fbce221 => generated 1266 bytes in 293 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:49:51.178955 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-8337d931-ad96-495a-a282-d2c765c0147b tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:49:51.181326 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-8337d931-ad96-495a-a282-d2c765c0147b tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] [instance: d49a602c-229d-48e6-8b26-c67f73ed8f2f] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:49:51.192537 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG dbcounter [-] [125641] Writing DB stats nova_cell0:SELECT=4 {{(pid=125641) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 00:49:51.197073 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG dbcounter [-] [125641] Writing DB stats nova_cell0:SELECT=18 {{(pid=125641) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 00:49:51.206019 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-8337d931-ad96-495a-a282-d2c765c0147b tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:49:51.207398 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-8337d931-ad96-495a-a282-d2c765c0147b tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:49:51.208221 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-8337d931-ad96-495a-a282-d2c765c0147b tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:49:51.209568 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-8337d931-ad96-495a-a282-d2c765c0147b tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:49:51.210291 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-8337d931-ad96-495a-a282-d2c765c0147b tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:49:51.211297 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-8337d931-ad96-495a-a282-d2c765c0147b tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:49:51.499293 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-8337d931-ad96-495a-a282-d2c765c0147b tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a48ed2d9ded741ca91afc1fbbd9c608f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '82d7723ef95b4c3faeac9835eeb233a5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:49:51.515770 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-8337d931-ad96-495a-a282-d2c765c0147b tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] 10.222.0.26 "GET /compute/v2.1/servers/d49a602c-229d-48e6-8b26-c67f73ed8f2f" status: 200 len: 1364 microversion: 2.1 time: 0.344295 Oct 19 00:49:51.517182 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 799/914] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:49:51 2022] GET /compute/v2.1/servers/d49a602c-229d-48e6-8b26-c67f73ed8f2f => generated 1364 bytes in 347 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 19 00:49:51.525896 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-972830f9-6290-4c7f-8069-e4e79aea861d tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:49:51.527101 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-972830f9-6290-4c7f-8069-e4e79aea861d tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] [instance: ae2afe94-f67b-419f-bd5f-aefc5fbce221] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:49:51.538953 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-972830f9-6290-4c7f-8069-e4e79aea861d tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:49:51.540394 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-972830f9-6290-4c7f-8069-e4e79aea861d tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:49:51.541756 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-972830f9-6290-4c7f-8069-e4e79aea861d tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:49:51.543164 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-972830f9-6290-4c7f-8069-e4e79aea861d tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:49:51.544153 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-972830f9-6290-4c7f-8069-e4e79aea861d tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:49:51.545038 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-972830f9-6290-4c7f-8069-e4e79aea861d tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:49:51.619819 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG dbcounter [-] [125641] Writing DB stats nova_cell1:SELECT=100 {{(pid=125641) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 00:49:51.933858 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-972830f9-6290-4c7f-8069-e4e79aea861d tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f738e67d8c2340028da1254b60d7810e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8684504b0eff415b9063cebae21ca667', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:49:51.950240 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-972830f9-6290-4c7f-8069-e4e79aea861d tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] 10.222.0.26 "GET /compute/v2.1/servers/ae2afe94-f67b-419f-bd5f-aefc5fbce221" status: 200 len: 1266 microversion: 2.1 time: 0.430948 Oct 19 00:49:51.964198 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 800/915] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:49:51 2022] GET /compute/v2.1/servers/ae2afe94-f67b-419f-bd5f-aefc5fbce221 => generated 1266 bytes in 446 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:49:52.543943 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-dc92aebb-4970-47be-98ec-02bdb9daeac7 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:49:52.545353 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-dc92aebb-4970-47be-98ec-02bdb9daeac7 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] [instance: d49a602c-229d-48e6-8b26-c67f73ed8f2f] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:49:52.559081 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-dc92aebb-4970-47be-98ec-02bdb9daeac7 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:49:52.560890 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-dc92aebb-4970-47be-98ec-02bdb9daeac7 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:49:52.562546 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-dc92aebb-4970-47be-98ec-02bdb9daeac7 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:49:52.564749 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-dc92aebb-4970-47be-98ec-02bdb9daeac7 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:49:52.566747 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-dc92aebb-4970-47be-98ec-02bdb9daeac7 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:49:52.567207 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-dc92aebb-4970-47be-98ec-02bdb9daeac7 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:49:52.884404 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-dc92aebb-4970-47be-98ec-02bdb9daeac7 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a48ed2d9ded741ca91afc1fbbd9c608f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '82d7723ef95b4c3faeac9835eeb233a5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:49:52.903671 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-dc92aebb-4970-47be-98ec-02bdb9daeac7 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] 10.222.0.26 "GET /compute/v2.1/servers/d49a602c-229d-48e6-8b26-c67f73ed8f2f" status: 200 len: 1364 microversion: 2.1 time: 0.364382 Oct 19 00:49:52.903671 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 801/916] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:49:52 2022] GET /compute/v2.1/servers/d49a602c-229d-48e6-8b26-c67f73ed8f2f => generated 1364 bytes in 367 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:49:53.003093 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-da46fde7-7461-48a2-9620-f14f7e853d86 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:49:53.004089 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-da46fde7-7461-48a2-9620-f14f7e853d86 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] [instance: ae2afe94-f67b-419f-bd5f-aefc5fbce221] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:49:53.012044 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-da46fde7-7461-48a2-9620-f14f7e853d86 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:49:53.012830 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-da46fde7-7461-48a2-9620-f14f7e853d86 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:49:53.013396 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-da46fde7-7461-48a2-9620-f14f7e853d86 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:49:53.014478 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-da46fde7-7461-48a2-9620-f14f7e853d86 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:49:53.015355 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-da46fde7-7461-48a2-9620-f14f7e853d86 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:49:53.015951 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-da46fde7-7461-48a2-9620-f14f7e853d86 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:49:53.244166 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-da46fde7-7461-48a2-9620-f14f7e853d86 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f738e67d8c2340028da1254b60d7810e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8684504b0eff415b9063cebae21ca667', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:49:53.261640 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-da46fde7-7461-48a2-9620-f14f7e853d86 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] 10.222.0.26 "GET /compute/v2.1/servers/ae2afe94-f67b-419f-bd5f-aefc5fbce221" status: 200 len: 1266 microversion: 2.1 time: 0.264540 Oct 19 00:49:53.298227 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 802/917] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:49:52 2022] GET /compute/v2.1/servers/ae2afe94-f67b-419f-bd5f-aefc5fbce221 => generated 1266 bytes in 283 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 19 00:49:53.801875 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [req-22ad3b21-cac6-46f5-83b7-1f873623316c req-f0b88db3-b5f5-4e89-813d-0f1d5f76beda service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "d49a602c-229d-48e6-8b26-c67f73ed8f2f", "name": "network-vif-plugged", "status": "completed", "tag": "a6830382-e592-4ea8-9d6a-63da173d5d01"}]} {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 19 00:49:53.842930 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-22ad3b21-cac6-46f5-83b7-1f873623316c req-f0b88db3-b5f5-4e89-813d-0f1d5f76beda service nova] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:49:53.844552 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-22ad3b21-cac6-46f5-83b7-1f873623316c req-f0b88db3-b5f5-4e89-813d-0f1d5f76beda service nova] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:49:53.846601 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-22ad3b21-cac6-46f5-83b7-1f873623316c req-f0b88db3-b5f5-4e89-813d-0f1d5f76beda service nova] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:49:53.882628 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.compute.server_external_events [req-22ad3b21-cac6-46f5-83b7-1f873623316c req-f0b88db3-b5f5-4e89-813d-0f1d5f76beda service nova] Creating event network-vif-plugged:a6830382-e592-4ea8-9d6a-63da173d5d01 for instance d49a602c-229d-48e6-8b26-c67f73ed8f2f on hv-cinder-maste Oct 19 00:49:53.893712 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [req-22ad3b21-cac6-46f5-83b7-1f873623316c req-f0b88db3-b5f5-4e89-813d-0f1d5f76beda service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.106489 Oct 19 00:49:53.895820 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 803/918] 10.222.0.26 () {64 vars in 1317 bytes} [Wed Oct 19 00:49:53 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 112 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 19 00:49:53.946244 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-2a137a87-5961-47a7-b4f5-384d23f54687 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:49:53.951626 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-2a137a87-5961-47a7-b4f5-384d23f54687 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] [instance: d49a602c-229d-48e6-8b26-c67f73ed8f2f] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:49:53.977279 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-2a137a87-5961-47a7-b4f5-384d23f54687 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:49:53.979028 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-2a137a87-5961-47a7-b4f5-384d23f54687 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:49:53.980155 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-2a137a87-5961-47a7-b4f5-384d23f54687 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:49:53.984317 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-2a137a87-5961-47a7-b4f5-384d23f54687 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:49:53.986213 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-2a137a87-5961-47a7-b4f5-384d23f54687 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:49:53.989535 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-2a137a87-5961-47a7-b4f5-384d23f54687 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:49:54.388921 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-2a137a87-5961-47a7-b4f5-384d23f54687 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a48ed2d9ded741ca91afc1fbbd9c608f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '82d7723ef95b4c3faeac9835eeb233a5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:49:54.426773 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-2a137a87-5961-47a7-b4f5-384d23f54687 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] 10.222.0.26 "GET /compute/v2.1/servers/d49a602c-229d-48e6-8b26-c67f73ed8f2f" status: 200 len: 1364 microversion: 2.1 time: 0.500972 Oct 19 00:49:54.428734 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 804/919] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:49:53 2022] GET /compute/v2.1/servers/d49a602c-229d-48e6-8b26-c67f73ed8f2f => generated 1364 bytes in 505 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:49:54.445843 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-994a3531-662a-4d13-a42b-2e51f071f26e tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:49:54.450030 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-994a3531-662a-4d13-a42b-2e51f071f26e tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] [instance: ae2afe94-f67b-419f-bd5f-aefc5fbce221] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:49:54.466258 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-994a3531-662a-4d13-a42b-2e51f071f26e tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:49:54.468222 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-994a3531-662a-4d13-a42b-2e51f071f26e tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:49:54.469542 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-994a3531-662a-4d13-a42b-2e51f071f26e tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:49:54.473724 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-994a3531-662a-4d13-a42b-2e51f071f26e tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:49:54.477329 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-994a3531-662a-4d13-a42b-2e51f071f26e tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:49:54.480398 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-994a3531-662a-4d13-a42b-2e51f071f26e tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:49:54.918570 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-994a3531-662a-4d13-a42b-2e51f071f26e tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f738e67d8c2340028da1254b60d7810e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8684504b0eff415b9063cebae21ca667', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:49:54.935783 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-994a3531-662a-4d13-a42b-2e51f071f26e tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] 10.222.0.26 "GET /compute/v2.1/servers/ae2afe94-f67b-419f-bd5f-aefc5fbce221" status: 200 len: 1266 microversion: 2.1 time: 0.504933 Oct 19 00:49:54.948329 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 805/920] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:49:54 2022] GET /compute/v2.1/servers/ae2afe94-f67b-419f-bd5f-aefc5fbce221 => generated 1266 bytes in 519 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:49:55.457596 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-dca0718a-8e2f-4e90-8806-f0703f6dd4a2 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:49:55.459140 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-dca0718a-8e2f-4e90-8806-f0703f6dd4a2 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] [instance: d49a602c-229d-48e6-8b26-c67f73ed8f2f] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:49:55.483841 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-dca0718a-8e2f-4e90-8806-f0703f6dd4a2 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:49:55.485123 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-dca0718a-8e2f-4e90-8806-f0703f6dd4a2 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:49:55.486161 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-dca0718a-8e2f-4e90-8806-f0703f6dd4a2 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:49:55.487924 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-dca0718a-8e2f-4e90-8806-f0703f6dd4a2 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:49:55.489019 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-dca0718a-8e2f-4e90-8806-f0703f6dd4a2 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:49:55.490024 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-dca0718a-8e2f-4e90-8806-f0703f6dd4a2 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:49:55.904254 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-dca0718a-8e2f-4e90-8806-f0703f6dd4a2 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a48ed2d9ded741ca91afc1fbbd9c608f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '82d7723ef95b4c3faeac9835eeb233a5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:49:55.930110 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-dca0718a-8e2f-4e90-8806-f0703f6dd4a2 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] 10.222.0.26 "GET /compute/v2.1/servers/d49a602c-229d-48e6-8b26-c67f73ed8f2f" status: 200 len: 1364 microversion: 2.1 time: 0.473076 Oct 19 00:49:55.947548 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 806/921] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:49:55 2022] GET /compute/v2.1/servers/d49a602c-229d-48e6-8b26-c67f73ed8f2f => generated 1364 bytes in 499 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 19 00:49:56.000442 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-8e7fafe2-b840-4a66-883b-99c7f589d0c4 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:49:56.003331 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-8e7fafe2-b840-4a66-883b-99c7f589d0c4 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] [instance: ae2afe94-f67b-419f-bd5f-aefc5fbce221] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:49:56.016695 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-8e7fafe2-b840-4a66-883b-99c7f589d0c4 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:49:56.018110 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-8e7fafe2-b840-4a66-883b-99c7f589d0c4 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:49:56.019643 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-8e7fafe2-b840-4a66-883b-99c7f589d0c4 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:49:56.021664 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-8e7fafe2-b840-4a66-883b-99c7f589d0c4 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:49:56.023269 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-8e7fafe2-b840-4a66-883b-99c7f589d0c4 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:49:56.024359 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-8e7fafe2-b840-4a66-883b-99c7f589d0c4 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:49:56.891201 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-8e7fafe2-b840-4a66-883b-99c7f589d0c4 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f738e67d8c2340028da1254b60d7810e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8684504b0eff415b9063cebae21ca667', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:49:56.909773 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-8e7fafe2-b840-4a66-883b-99c7f589d0c4 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] 10.222.0.26 "GET /compute/v2.1/servers/ae2afe94-f67b-419f-bd5f-aefc5fbce221" status: 200 len: 1266 microversion: 2.1 time: 0.926601 Oct 19 00:49:56.926524 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 807/922] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:49:55 2022] GET /compute/v2.1/servers/ae2afe94-f67b-419f-bd5f-aefc5fbce221 => generated 1266 bytes in 939 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:49:56.971837 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-7566d1af-08a5-4315-998b-0ad424226c5b tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:49:56.972823 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-7566d1af-08a5-4315-998b-0ad424226c5b tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] [instance: d49a602c-229d-48e6-8b26-c67f73ed8f2f] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:49:56.979681 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-7566d1af-08a5-4315-998b-0ad424226c5b tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:49:56.980590 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-7566d1af-08a5-4315-998b-0ad424226c5b tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:49:56.981345 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-7566d1af-08a5-4315-998b-0ad424226c5b tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:49:56.982325 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-7566d1af-08a5-4315-998b-0ad424226c5b tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:49:56.983261 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-7566d1af-08a5-4315-998b-0ad424226c5b tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:49:56.984006 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-7566d1af-08a5-4315-998b-0ad424226c5b tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:49:57.243222 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-7566d1af-08a5-4315-998b-0ad424226c5b tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a48ed2d9ded741ca91afc1fbbd9c608f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '82d7723ef95b4c3faeac9835eeb233a5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:49:57.255660 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-7566d1af-08a5-4315-998b-0ad424226c5b tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] 10.222.0.26 "GET /compute/v2.1/servers/d49a602c-229d-48e6-8b26-c67f73ed8f2f" status: 200 len: 1549 microversion: 2.1 time: 0.290694 Oct 19 00:49:57.270572 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 808/923] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:49:56 2022] GET /compute/v2.1/servers/d49a602c-229d-48e6-8b26-c67f73ed8f2f => generated 1549 bytes in 292 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:49:57.301663 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-553c5de0-0cf4-4567-bfd4-e9e008e5a5bd tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:49:57.302967 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-553c5de0-0cf4-4567-bfd4-e9e008e5a5bd tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] [instance: d49a602c-229d-48e6-8b26-c67f73ed8f2f] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:49:57.317701 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-553c5de0-0cf4-4567-bfd4-e9e008e5a5bd tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:49:57.319003 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-553c5de0-0cf4-4567-bfd4-e9e008e5a5bd tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:49:57.320387 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-553c5de0-0cf4-4567-bfd4-e9e008e5a5bd tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:49:57.321969 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-553c5de0-0cf4-4567-bfd4-e9e008e5a5bd tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:49:57.323127 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-553c5de0-0cf4-4567-bfd4-e9e008e5a5bd tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:49:57.324083 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-553c5de0-0cf4-4567-bfd4-e9e008e5a5bd tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:49:57.566520 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-553c5de0-0cf4-4567-bfd4-e9e008e5a5bd tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] 10.222.0.26 "GET /compute/v2.1/servers/d49a602c-229d-48e6-8b26-c67f73ed8f2f/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.273574 Oct 19 00:49:57.570615 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 809/924] 10.222.0.26 () {58 vars in 1275 bytes} [Wed Oct 19 00:49:57 2022] GET /compute/v2.1/servers/d49a602c-229d-48e6-8b26-c67f73ed8f2f/os-interface => generated 284 bytes in 276 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 19 00:49:57.953710 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-a297f178-a04f-4f96-a7ed-713788f93828 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:49:57.955342 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-a297f178-a04f-4f96-a7ed-713788f93828 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] [instance: ae2afe94-f67b-419f-bd5f-aefc5fbce221] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:49:57.966759 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-a297f178-a04f-4f96-a7ed-713788f93828 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:49:57.968132 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-a297f178-a04f-4f96-a7ed-713788f93828 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:49:57.969793 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-a297f178-a04f-4f96-a7ed-713788f93828 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:49:57.971435 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-a297f178-a04f-4f96-a7ed-713788f93828 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:49:57.972570 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-a297f178-a04f-4f96-a7ed-713788f93828 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:49:57.974052 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-a297f178-a04f-4f96-a7ed-713788f93828 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:49:58.494467 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-a297f178-a04f-4f96-a7ed-713788f93828 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f738e67d8c2340028da1254b60d7810e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8684504b0eff415b9063cebae21ca667', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:49:58.527304 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-a297f178-a04f-4f96-a7ed-713788f93828 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] 10.222.0.26 "GET /compute/v2.1/servers/ae2afe94-f67b-419f-bd5f-aefc5fbce221" status: 200 len: 1266 microversion: 2.1 time: 0.581899 Oct 19 00:49:58.544168 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 810/925] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:49:57 2022] GET /compute/v2.1/servers/ae2afe94-f67b-419f-bd5f-aefc5fbce221 => generated 1266 bytes in 600 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:49:59.299969 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [req-d2092c9a-0d94-4ee4-bb22-32ae6c6bdfa7 req-513355e4-3c25-4cb4-865b-2137007e5018 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "d49a602c-229d-48e6-8b26-c67f73ed8f2f", "tag": "a6830382-e592-4ea8-9d6a-63da173d5d01"}]} {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 19 00:49:59.325116 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-d2092c9a-0d94-4ee4-bb22-32ae6c6bdfa7 req-513355e4-3c25-4cb4-865b-2137007e5018 service nova] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:49:59.325604 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-d2092c9a-0d94-4ee4-bb22-32ae6c6bdfa7 req-513355e4-3c25-4cb4-865b-2137007e5018 service nova] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:49:59.326038 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-d2092c9a-0d94-4ee4-bb22-32ae6c6bdfa7 req-513355e4-3c25-4cb4-865b-2137007e5018 service nova] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:49:59.360924 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.compute.server_external_events [req-d2092c9a-0d94-4ee4-bb22-32ae6c6bdfa7 req-513355e4-3c25-4cb4-865b-2137007e5018 service nova] Creating event network-changed:a6830382-e592-4ea8-9d6a-63da173d5d01 for instance d49a602c-229d-48e6-8b26-c67f73ed8f2f on hv-cinder-maste Oct 19 00:49:59.374460 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [req-d2092c9a-0d94-4ee4-bb22-32ae6c6bdfa7 req-513355e4-3c25-4cb4-865b-2137007e5018 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.078409 Oct 19 00:49:59.378875 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 811/926] 10.222.0.26 () {64 vars in 1317 bytes} [Wed Oct 19 00:49:59 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 19 00:49:59.585799 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-c0538c05-3518-43b8-8c8f-48b7b6f7b98f tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:49:59.587163 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-c0538c05-3518-43b8-8c8f-48b7b6f7b98f tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] [instance: ae2afe94-f67b-419f-bd5f-aefc5fbce221] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:49:59.617649 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-c0538c05-3518-43b8-8c8f-48b7b6f7b98f tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:49:59.622839 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-c0538c05-3518-43b8-8c8f-48b7b6f7b98f tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:49:59.629510 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-c0538c05-3518-43b8-8c8f-48b7b6f7b98f tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:49:59.632795 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-c0538c05-3518-43b8-8c8f-48b7b6f7b98f tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:49:59.635070 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-c0538c05-3518-43b8-8c8f-48b7b6f7b98f tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:49:59.636360 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-c0538c05-3518-43b8-8c8f-48b7b6f7b98f tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:50:00.027906 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-c0538c05-3518-43b8-8c8f-48b7b6f7b98f tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f738e67d8c2340028da1254b60d7810e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8684504b0eff415b9063cebae21ca667', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:50:00.055542 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-c0538c05-3518-43b8-8c8f-48b7b6f7b98f tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] 10.222.0.26 "GET /compute/v2.1/servers/ae2afe94-f67b-419f-bd5f-aefc5fbce221" status: 200 len: 1266 microversion: 2.1 time: 0.483451 Oct 19 00:50:00.056148 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 812/927] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:49:59 2022] GET /compute/v2.1/servers/ae2afe94-f67b-419f-bd5f-aefc5fbce221 => generated 1266 bytes in 486 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:50:00.275152 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [req-683dede2-ef90-4427-b988-f4d147bcaac1 req-37add58d-754f-4645-84d0-9495ca91932d service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "ae2afe94-f67b-419f-bd5f-aefc5fbce221", "name": "network-vif-plugged", "status": "completed", "tag": "c76271cc-707a-4a90-9ae9-135461e8154f"}]} {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 19 00:50:00.310186 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-683dede2-ef90-4427-b988-f4d147bcaac1 req-37add58d-754f-4645-84d0-9495ca91932d service nova] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:50:00.315367 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-683dede2-ef90-4427-b988-f4d147bcaac1 req-37add58d-754f-4645-84d0-9495ca91932d service nova] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:50:00.316235 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-683dede2-ef90-4427-b988-f4d147bcaac1 req-37add58d-754f-4645-84d0-9495ca91932d service nova] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:50:00.367896 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.compute.server_external_events [req-683dede2-ef90-4427-b988-f4d147bcaac1 req-37add58d-754f-4645-84d0-9495ca91932d service nova] Creating event network-vif-plugged:c76271cc-707a-4a90-9ae9-135461e8154f for instance ae2afe94-f67b-419f-bd5f-aefc5fbce221 on hv-cinder-maste Oct 19 00:50:00.392787 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [req-683dede2-ef90-4427-b988-f4d147bcaac1 req-37add58d-754f-4645-84d0-9495ca91932d service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.124560 Oct 19 00:50:00.400329 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 813/928] 10.222.0.26 () {64 vars in 1317 bytes} [Wed Oct 19 00:50:00 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 133 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 19 00:50:01.103617 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-9e7e1a1e-dace-435f-ba3a-7f66f64d85f9 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:50:01.108902 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-9e7e1a1e-dace-435f-ba3a-7f66f64d85f9 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] [instance: ae2afe94-f67b-419f-bd5f-aefc5fbce221] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:50:01.122001 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-9e7e1a1e-dace-435f-ba3a-7f66f64d85f9 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:50:01.122949 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-9e7e1a1e-dace-435f-ba3a-7f66f64d85f9 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:50:01.123727 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-9e7e1a1e-dace-435f-ba3a-7f66f64d85f9 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:50:01.125229 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-9e7e1a1e-dace-435f-ba3a-7f66f64d85f9 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:50:01.127463 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-9e7e1a1e-dace-435f-ba3a-7f66f64d85f9 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:50:01.128668 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-9e7e1a1e-dace-435f-ba3a-7f66f64d85f9 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:50:01.414264 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-9e7e1a1e-dace-435f-ba3a-7f66f64d85f9 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f738e67d8c2340028da1254b60d7810e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8684504b0eff415b9063cebae21ca667', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:50:01.423947 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-9e7e1a1e-dace-435f-ba3a-7f66f64d85f9 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] 10.222.0.26 "GET /compute/v2.1/servers/ae2afe94-f67b-419f-bd5f-aefc5fbce221" status: 200 len: 1266 microversion: 2.1 time: 0.329123 Oct 19 00:50:01.429887 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 814/929] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:50:01 2022] GET /compute/v2.1/servers/ae2afe94-f67b-419f-bd5f-aefc5fbce221 => generated 1266 bytes in 337 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:50:02.461970 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-e20b5e1a-2505-4351-ac1c-dbca3cef1a26 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:50:02.467109 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-e20b5e1a-2505-4351-ac1c-dbca3cef1a26 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] [instance: ae2afe94-f67b-419f-bd5f-aefc5fbce221] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:50:02.485698 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-e20b5e1a-2505-4351-ac1c-dbca3cef1a26 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:50:02.487407 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-e20b5e1a-2505-4351-ac1c-dbca3cef1a26 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:50:02.488595 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-e20b5e1a-2505-4351-ac1c-dbca3cef1a26 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:50:02.490121 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-e20b5e1a-2505-4351-ac1c-dbca3cef1a26 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:50:02.491269 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-e20b5e1a-2505-4351-ac1c-dbca3cef1a26 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:50:02.492295 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-e20b5e1a-2505-4351-ac1c-dbca3cef1a26 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:50:02.772394 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-e20b5e1a-2505-4351-ac1c-dbca3cef1a26 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f738e67d8c2340028da1254b60d7810e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8684504b0eff415b9063cebae21ca667', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:50:02.788132 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-e20b5e1a-2505-4351-ac1c-dbca3cef1a26 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] 10.222.0.26 "GET /compute/v2.1/servers/ae2afe94-f67b-419f-bd5f-aefc5fbce221" status: 200 len: 1453 microversion: 2.1 time: 0.336352 Oct 19 00:50:02.789030 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 815/930] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:50:02 2022] GET /compute/v2.1/servers/ae2afe94-f67b-419f-bd5f-aefc5fbce221 => generated 1453 bytes in 339 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:50:02.828795 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-448b7075-6557-47dd-9a3e-26db3eb9913c tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:50:02.829947 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-448b7075-6557-47dd-9a3e-26db3eb9913c tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] [instance: ae2afe94-f67b-419f-bd5f-aefc5fbce221] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:50:02.840432 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-448b7075-6557-47dd-9a3e-26db3eb9913c tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:50:02.841268 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-448b7075-6557-47dd-9a3e-26db3eb9913c tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:50:02.841938 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-448b7075-6557-47dd-9a3e-26db3eb9913c tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:50:02.843534 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-448b7075-6557-47dd-9a3e-26db3eb9913c tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:50:02.845569 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-448b7075-6557-47dd-9a3e-26db3eb9913c tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:50:02.846526 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-448b7075-6557-47dd-9a3e-26db3eb9913c tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:50:03.033198 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-448b7075-6557-47dd-9a3e-26db3eb9913c tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f738e67d8c2340028da1254b60d7810e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8684504b0eff415b9063cebae21ca667', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:50:03.044548 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-448b7075-6557-47dd-9a3e-26db3eb9913c tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] 10.222.0.26 "GET /compute/v2.1/servers/ae2afe94-f67b-419f-bd5f-aefc5fbce221" status: 200 len: 1453 microversion: 2.1 time: 0.221725 Oct 19 00:50:03.046071 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 816/931] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:50:02 2022] GET /compute/v2.1/servers/ae2afe94-f67b-419f-bd5f-aefc5fbce221 => generated 1453 bytes in 224 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:50:06.524162 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [req-655bab78-d703-4a0e-b133-1907d103fdf5 req-dd5a990e-2d11-45e0-ac24-64192455498a service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "ae2afe94-f67b-419f-bd5f-aefc5fbce221", "tag": "c76271cc-707a-4a90-9ae9-135461e8154f"}]} {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 19 00:50:06.536500 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-655bab78-d703-4a0e-b133-1907d103fdf5 req-dd5a990e-2d11-45e0-ac24-64192455498a service nova] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:50:06.537270 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-655bab78-d703-4a0e-b133-1907d103fdf5 req-dd5a990e-2d11-45e0-ac24-64192455498a service nova] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:50:06.537867 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-655bab78-d703-4a0e-b133-1907d103fdf5 req-dd5a990e-2d11-45e0-ac24-64192455498a service nova] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:50:06.576081 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.compute.server_external_events [req-655bab78-d703-4a0e-b133-1907d103fdf5 req-dd5a990e-2d11-45e0-ac24-64192455498a service nova] Creating event network-changed:c76271cc-707a-4a90-9ae9-135461e8154f for instance ae2afe94-f67b-419f-bd5f-aefc5fbce221 on hv-cinder-maste Oct 19 00:50:06.583436 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [req-655bab78-d703-4a0e-b133-1907d103fdf5 req-dd5a990e-2d11-45e0-ac24-64192455498a service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.062684 Oct 19 00:50:06.585481 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 817/932] 10.222.0.26 () {64 vars in 1317 bytes} [Wed Oct 19 00:50:06 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 68 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 19 00:50:13.993908 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-fa52d638-22bb-4ace-b77d-84c5f3c40abb tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Action: 'create', calling method: .version_select at 0x7f71cb5d4310>, body: {"keypair": {"name": "tempest-TestMinimumBasicScenario-863658547", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBFof+LVaDLtuRc0Q4UDx7A8DZ/+sG6t9xSELb6CqOuBGEOZhjCc6Cxrfzj/WXDnRl/k+xfqUvRPG+EzYeGV9c/N9b6ZlhQG5n+nRqc6XbKCbo0LgaH7S21TlKFcA4vH+qQ=="}} {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 19 00:50:14.011898 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.quota [None req-fa52d638-22bb-4ace-b77d-84c5f3c40abb tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Getting quotas for project a8a6d900d9d344a694b955f51fa46aa9. Resources: {'key_pairs'} {{(pid=125641) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 19 00:50:14.017119 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.quota [None req-fa52d638-22bb-4ace-b77d-84c5f3c40abb tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Getting quotas for user 531efd120096452caf8c0b7a7f6b9ba1 and project a8a6d900d9d344a694b955f51fa46aa9. Resources: {'key_pairs'} {{(pid=125641) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 19 00:50:14.044846 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-fa52d638-22bb-4ace-b77d-84c5f3c40abb tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] 10.222.0.26 "POST /compute/v2.1/os-keypairs" status: 200 len: 402 microversion: 2.1 time: 0.055612 Oct 19 00:50:14.046025 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 818/933] 10.222.0.26 () {60 vars in 1159 bytes} [Wed Oct 19 00:50:13 2022] POST /compute/v2.1/os-keypairs => generated 402 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 19 00:50:14.101956 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-4ce3ccd3-8a7f-4cf5-895f-36637c7322c2 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestMinimumBasicScenario-server-1553025449", "imageRef": "9ff36141-cfc2-4843-b24d-031a0c880a60", "flavorRef": "452", "key_name": "tempest-TestMinimumBasicScenario-863658547", "networks": [{"uuid": "55e165e6-4b57-4574-ad76-be38ba32d98e"}]}} {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 19 00:50:14.162737 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.network.neutron [None req-4ce3ccd3-8a7f-4cf5-895f-36637c7322c2 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] validate_networks() for [('55e165e6-4b57-4574-ad76-be38ba32d98e', None, None, None, None, None)] {{(pid=125641) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Oct 19 00:50:14.621905 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.virt.hardware [None req-4ce3ccd3-8a7f-4cf5-895f-36637c7322c2 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Flavor limits 0:0:0 {{(pid=125641) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Oct 19 00:50:14.622465 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.virt.hardware [None req-4ce3ccd3-8a7f-4cf5-895f-36637c7322c2 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Image limits 0:0:0 {{(pid=125641) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Oct 19 00:50:14.623489 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.virt.hardware [None req-4ce3ccd3-8a7f-4cf5-895f-36637c7322c2 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Flavor pref 0:0:0 {{(pid=125641) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Oct 19 00:50:14.624024 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.virt.hardware [None req-4ce3ccd3-8a7f-4cf5-895f-36637c7322c2 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Image pref 0:0:0 {{(pid=125641) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Oct 19 00:50:14.624540 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.virt.hardware [None req-4ce3ccd3-8a7f-4cf5-895f-36637c7322c2 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=125641) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Oct 19 00:50:14.634845 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-4ce3ccd3-8a7f-4cf5-895f-36637c7322c2 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:50:14.635381 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-4ce3ccd3-8a7f-4cf5-895f-36637c7322c2 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:50:14.635914 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-4ce3ccd3-8a7f-4cf5-895f-36637c7322c2 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:50:14.636771 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-4ce3ccd3-8a7f-4cf5-895f-36637c7322c2 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:50:14.637332 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-4ce3ccd3-8a7f-4cf5-895f-36637c7322c2 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:50:14.637951 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-4ce3ccd3-8a7f-4cf5-895f-36637c7322c2 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:50:14.659138 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.quota [None req-4ce3ccd3-8a7f-4cf5-895f-36637c7322c2 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Getting quotas for project a8a6d900d9d344a694b955f51fa46aa9. Resources: {'cores', 'ram', 'instances'} {{(pid=125641) _get_quotas /opt/stack/nova/nova/quota.py:393}} Oct 19 00:50:14.664154 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.quota [None req-4ce3ccd3-8a7f-4cf5-895f-36637c7322c2 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Getting quotas for user 531efd120096452caf8c0b7a7f6b9ba1 and project a8a6d900d9d344a694b955f51fa46aa9. Resources: {'cores', 'ram', 'instances'} {{(pid=125641) _get_quotas /opt/stack/nova/nova/quota.py:383}} Oct 19 00:50:14.678253 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-4ce3ccd3-8a7f-4cf5-895f-36637c7322c2 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Going to run 1 instances... {{(pid=125641) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Oct 19 00:50:14.680807 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-4ce3ccd3-8a7f-4cf5-895f-36637c7322c2 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] [instance: bfbf8641-41b8-47d8-af4d-ef17c6bd1935] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='9ff36141-cfc2-4843-b24d-031a0c880a60',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=125641) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Oct 19 00:50:14.738477 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-4ce3ccd3-8a7f-4cf5-895f-36637c7322c2 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] 10.222.0.26 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.640547 Oct 19 00:50:14.739394 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 819/934] 10.222.0.26 () {60 vars in 1147 bytes} [Wed Oct 19 00:50:14 2022] POST /compute/v2.1/servers => generated 380 bytes in 642 msecs (HTTP/1.1 202) 10 headers in 451 bytes (2 switches on core 0) Oct 19 00:50:14.751702 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-bf22c70c-d613-4306-bf03-8ff80ec64c7b tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:50:14.752705 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-bf22c70c-d613-4306-bf03-8ff80ec64c7b tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] [instance: bfbf8641-41b8-47d8-af4d-ef17c6bd1935] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:50:14.836670 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-bf22c70c-d613-4306-bf03-8ff80ec64c7b tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '531efd120096452caf8c0b7a7f6b9ba1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a8a6d900d9d344a694b955f51fa46aa9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:50:14.846051 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-bf22c70c-d613-4306-bf03-8ff80ec64c7b tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] 10.222.0.26 "GET /compute/v2.1/servers/bfbf8641-41b8-47d8-af4d-ef17c6bd1935" status: 200 len: 1258 microversion: 2.1 time: 0.098929 Oct 19 00:50:14.846707 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 820/935] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:50:14 2022] GET /compute/v2.1/servers/bfbf8641-41b8-47d8-af4d-ef17c6bd1935 => generated 1258 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:50:15.876540 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-d1604189-9190-4fb2-9982-710d4f106112 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:50:15.877375 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-d1604189-9190-4fb2-9982-710d4f106112 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] [instance: bfbf8641-41b8-47d8-af4d-ef17c6bd1935] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:50:15.885518 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d1604189-9190-4fb2-9982-710d4f106112 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:50:15.886151 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d1604189-9190-4fb2-9982-710d4f106112 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:50:15.886758 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d1604189-9190-4fb2-9982-710d4f106112 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:50:15.887590 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d1604189-9190-4fb2-9982-710d4f106112 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:50:15.888140 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d1604189-9190-4fb2-9982-710d4f106112 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:50:15.889335 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d1604189-9190-4fb2-9982-710d4f106112 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:50:15.957515 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-d1604189-9190-4fb2-9982-710d4f106112 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '531efd120096452caf8c0b7a7f6b9ba1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a8a6d900d9d344a694b955f51fa46aa9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:50:15.965989 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-d1604189-9190-4fb2-9982-710d4f106112 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] 10.222.0.26 "GET /compute/v2.1/servers/bfbf8641-41b8-47d8-af4d-ef17c6bd1935" status: 200 len: 1310 microversion: 2.1 time: 0.093910 Oct 19 00:50:15.966613 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 821/936] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:50:15 2022] GET /compute/v2.1/servers/bfbf8641-41b8-47d8-af4d-ef17c6bd1935 => generated 1310 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 19 00:50:17.005702 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-981afaa8-5d79-4001-9fa6-ed6a4684ae41 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:50:17.010061 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-981afaa8-5d79-4001-9fa6-ed6a4684ae41 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] [instance: bfbf8641-41b8-47d8-af4d-ef17c6bd1935] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:50:17.042865 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-981afaa8-5d79-4001-9fa6-ed6a4684ae41 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:50:17.043799 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-981afaa8-5d79-4001-9fa6-ed6a4684ae41 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.012s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:50:17.044464 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-981afaa8-5d79-4001-9fa6-ed6a4684ae41 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:50:17.046173 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-981afaa8-5d79-4001-9fa6-ed6a4684ae41 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:50:17.047018 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-981afaa8-5d79-4001-9fa6-ed6a4684ae41 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:50:17.047788 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-981afaa8-5d79-4001-9fa6-ed6a4684ae41 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:50:17.212636 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-981afaa8-5d79-4001-9fa6-ed6a4684ae41 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '531efd120096452caf8c0b7a7f6b9ba1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a8a6d900d9d344a694b955f51fa46aa9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:50:17.224653 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG dbcounter [-] [125641] Writing DB stats nova_cell1:SELECT=100 {{(pid=125641) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 00:50:17.230665 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-981afaa8-5d79-4001-9fa6-ed6a4684ae41 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] 10.222.0.26 "GET /compute/v2.1/servers/bfbf8641-41b8-47d8-af4d-ef17c6bd1935" status: 200 len: 1316 microversion: 2.1 time: 0.232291 Oct 19 00:50:17.232927 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 822/937] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:50:16 2022] GET /compute/v2.1/servers/bfbf8641-41b8-47d8-af4d-ef17c6bd1935 => generated 1316 bytes in 235 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 19 00:50:18.258313 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-d25dff22-ca78-4f6a-a076-f67abcd0907b tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:50:18.259567 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-d25dff22-ca78-4f6a-a076-f67abcd0907b tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] [instance: bfbf8641-41b8-47d8-af4d-ef17c6bd1935] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:50:18.272205 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d25dff22-ca78-4f6a-a076-f67abcd0907b tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:50:18.273863 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d25dff22-ca78-4f6a-a076-f67abcd0907b tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:50:18.275313 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d25dff22-ca78-4f6a-a076-f67abcd0907b tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:50:18.277023 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d25dff22-ca78-4f6a-a076-f67abcd0907b tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:50:18.278130 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d25dff22-ca78-4f6a-a076-f67abcd0907b tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:50:18.279386 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d25dff22-ca78-4f6a-a076-f67abcd0907b tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:50:18.532358 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-d25dff22-ca78-4f6a-a076-f67abcd0907b tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '531efd120096452caf8c0b7a7f6b9ba1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a8a6d900d9d344a694b955f51fa46aa9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:50:18.545336 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-d25dff22-ca78-4f6a-a076-f67abcd0907b tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] 10.222.0.26 "GET /compute/v2.1/servers/bfbf8641-41b8-47d8-af4d-ef17c6bd1935" status: 200 len: 1358 microversion: 2.1 time: 0.292563 Oct 19 00:50:18.546418 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 823/938] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:50:18 2022] GET /compute/v2.1/servers/bfbf8641-41b8-47d8-af4d-ef17c6bd1935 => generated 1358 bytes in 294 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 19 00:50:19.583009 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-067e6d29-aa8b-4483-ae64-2e7ec9f608df tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:50:19.584615 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-067e6d29-aa8b-4483-ae64-2e7ec9f608df tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] [instance: bfbf8641-41b8-47d8-af4d-ef17c6bd1935] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:50:19.589318 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG dbcounter [-] [125641] Writing DB stats nova_api:SELECT=80,nova_api:INSERT=4 {{(pid=125641) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 00:50:19.600056 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-067e6d29-aa8b-4483-ae64-2e7ec9f608df tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:50:19.600975 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-067e6d29-aa8b-4483-ae64-2e7ec9f608df tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:50:19.601801 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-067e6d29-aa8b-4483-ae64-2e7ec9f608df tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:50:19.603379 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-067e6d29-aa8b-4483-ae64-2e7ec9f608df tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:50:19.604137 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-067e6d29-aa8b-4483-ae64-2e7ec9f608df tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:50:19.604866 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-067e6d29-aa8b-4483-ae64-2e7ec9f608df tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:50:19.850652 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-067e6d29-aa8b-4483-ae64-2e7ec9f608df tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '531efd120096452caf8c0b7a7f6b9ba1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a8a6d900d9d344a694b955f51fa46aa9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:50:19.868128 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-067e6d29-aa8b-4483-ae64-2e7ec9f608df tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] 10.222.0.26 "GET /compute/v2.1/servers/bfbf8641-41b8-47d8-af4d-ef17c6bd1935" status: 200 len: 1358 microversion: 2.1 time: 0.293990 Oct 19 00:50:19.874728 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 824/939] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:50:19 2022] GET /compute/v2.1/servers/bfbf8641-41b8-47d8-af4d-ef17c6bd1935 => generated 1358 bytes in 302 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 19 00:50:20.916794 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-2e6ff92b-5068-431b-a96c-7e66a996b4b9 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:50:20.919435 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-2e6ff92b-5068-431b-a96c-7e66a996b4b9 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] [instance: bfbf8641-41b8-47d8-af4d-ef17c6bd1935] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:50:20.929343 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-2e6ff92b-5068-431b-a96c-7e66a996b4b9 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:50:20.930102 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-2e6ff92b-5068-431b-a96c-7e66a996b4b9 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:50:20.930756 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-2e6ff92b-5068-431b-a96c-7e66a996b4b9 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:50:20.931620 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-2e6ff92b-5068-431b-a96c-7e66a996b4b9 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:50:20.932222 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-2e6ff92b-5068-431b-a96c-7e66a996b4b9 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:50:20.932738 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-2e6ff92b-5068-431b-a96c-7e66a996b4b9 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:50:21.120435 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-2e6ff92b-5068-431b-a96c-7e66a996b4b9 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '531efd120096452caf8c0b7a7f6b9ba1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a8a6d900d9d344a694b955f51fa46aa9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:50:21.129629 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-2e6ff92b-5068-431b-a96c-7e66a996b4b9 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] 10.222.0.26 "GET /compute/v2.1/servers/bfbf8641-41b8-47d8-af4d-ef17c6bd1935" status: 200 len: 1358 microversion: 2.1 time: 0.221504 Oct 19 00:50:21.130617 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 825/940] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:50:20 2022] GET /compute/v2.1/servers/bfbf8641-41b8-47d8-af4d-ef17c6bd1935 => generated 1358 bytes in 224 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:50:22.172688 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-d688bf42-c8d2-4bab-8946-47349ea5d9a8 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:50:22.174658 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-d688bf42-c8d2-4bab-8946-47349ea5d9a8 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] [instance: bfbf8641-41b8-47d8-af4d-ef17c6bd1935] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:50:22.189126 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d688bf42-c8d2-4bab-8946-47349ea5d9a8 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:50:22.189863 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d688bf42-c8d2-4bab-8946-47349ea5d9a8 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:50:22.190692 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d688bf42-c8d2-4bab-8946-47349ea5d9a8 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:50:22.191896 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d688bf42-c8d2-4bab-8946-47349ea5d9a8 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:50:22.192660 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d688bf42-c8d2-4bab-8946-47349ea5d9a8 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:50:22.193408 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d688bf42-c8d2-4bab-8946-47349ea5d9a8 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:50:22.460745 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-d688bf42-c8d2-4bab-8946-47349ea5d9a8 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '531efd120096452caf8c0b7a7f6b9ba1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a8a6d900d9d344a694b955f51fa46aa9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:50:22.477001 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-d688bf42-c8d2-4bab-8946-47349ea5d9a8 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] 10.222.0.26 "GET /compute/v2.1/servers/bfbf8641-41b8-47d8-af4d-ef17c6bd1935" status: 200 len: 1358 microversion: 2.1 time: 0.321976 Oct 19 00:50:22.478086 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 826/941] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:50:22 2022] GET /compute/v2.1/servers/bfbf8641-41b8-47d8-af4d-ef17c6bd1935 => generated 1358 bytes in 326 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:50:23.514963 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-e3f1ef1c-0e69-4bef-a0b4-8bc7b775c494 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:50:23.516116 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-e3f1ef1c-0e69-4bef-a0b4-8bc7b775c494 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] [instance: bfbf8641-41b8-47d8-af4d-ef17c6bd1935] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:50:23.527697 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-e3f1ef1c-0e69-4bef-a0b4-8bc7b775c494 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:50:23.528509 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-e3f1ef1c-0e69-4bef-a0b4-8bc7b775c494 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:50:23.529381 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-e3f1ef1c-0e69-4bef-a0b4-8bc7b775c494 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:50:23.530711 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-e3f1ef1c-0e69-4bef-a0b4-8bc7b775c494 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:50:23.531545 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-e3f1ef1c-0e69-4bef-a0b4-8bc7b775c494 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:50:23.532246 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-e3f1ef1c-0e69-4bef-a0b4-8bc7b775c494 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:50:23.741874 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-e3f1ef1c-0e69-4bef-a0b4-8bc7b775c494 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '531efd120096452caf8c0b7a7f6b9ba1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a8a6d900d9d344a694b955f51fa46aa9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:50:23.751078 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-e3f1ef1c-0e69-4bef-a0b4-8bc7b775c494 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] 10.222.0.26 "GET /compute/v2.1/servers/bfbf8641-41b8-47d8-af4d-ef17c6bd1935" status: 200 len: 1358 microversion: 2.1 time: 0.242799 Oct 19 00:50:23.751685 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 827/942] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:50:23 2022] GET /compute/v2.1/servers/bfbf8641-41b8-47d8-af4d-ef17c6bd1935 => generated 1358 bytes in 244 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:50:24.781491 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-02ed551f-8bdf-458f-b679-acf605a268ac tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:50:24.782959 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-02ed551f-8bdf-458f-b679-acf605a268ac tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] [instance: bfbf8641-41b8-47d8-af4d-ef17c6bd1935] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:50:24.795199 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG dbcounter [-] [125641] Writing DB stats nova_cell0:SELECT=2 {{(pid=125641) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 00:50:24.800605 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-02ed551f-8bdf-458f-b679-acf605a268ac tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:50:24.802163 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-02ed551f-8bdf-458f-b679-acf605a268ac tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:50:24.803498 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-02ed551f-8bdf-458f-b679-acf605a268ac tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:50:24.804918 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-02ed551f-8bdf-458f-b679-acf605a268ac tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:50:24.805654 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-02ed551f-8bdf-458f-b679-acf605a268ac tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:50:24.806523 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-02ed551f-8bdf-458f-b679-acf605a268ac tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:50:24.844738 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG dbcounter [-] [125641] Writing DB stats nova_cell0:SELECT=12 {{(pid=125641) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 00:50:25.203025 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-02ed551f-8bdf-458f-b679-acf605a268ac tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '531efd120096452caf8c0b7a7f6b9ba1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a8a6d900d9d344a694b955f51fa46aa9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:50:25.215261 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-02ed551f-8bdf-458f-b679-acf605a268ac tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] 10.222.0.26 "GET /compute/v2.1/servers/bfbf8641-41b8-47d8-af4d-ef17c6bd1935" status: 200 len: 1358 microversion: 2.1 time: 0.442058 Oct 19 00:50:25.216047 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 828/943] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:50:24 2022] GET /compute/v2.1/servers/bfbf8641-41b8-47d8-af4d-ef17c6bd1935 => generated 1358 bytes in 444 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:50:26.240827 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-eac101b7-d4c7-450f-8b97-4be1ecfcf188 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:50:26.241967 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-eac101b7-d4c7-450f-8b97-4be1ecfcf188 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] [instance: bfbf8641-41b8-47d8-af4d-ef17c6bd1935] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:50:26.250699 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-eac101b7-d4c7-450f-8b97-4be1ecfcf188 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:50:26.251319 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-eac101b7-d4c7-450f-8b97-4be1ecfcf188 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:50:26.251953 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-eac101b7-d4c7-450f-8b97-4be1ecfcf188 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:50:26.252889 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-eac101b7-d4c7-450f-8b97-4be1ecfcf188 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:50:26.253417 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-eac101b7-d4c7-450f-8b97-4be1ecfcf188 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:50:26.254366 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-eac101b7-d4c7-450f-8b97-4be1ecfcf188 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:50:26.468060 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-eac101b7-d4c7-450f-8b97-4be1ecfcf188 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '531efd120096452caf8c0b7a7f6b9ba1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a8a6d900d9d344a694b955f51fa46aa9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:50:26.478116 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-eac101b7-d4c7-450f-8b97-4be1ecfcf188 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] 10.222.0.26 "GET /compute/v2.1/servers/bfbf8641-41b8-47d8-af4d-ef17c6bd1935" status: 200 len: 1358 microversion: 2.1 time: 0.241492 Oct 19 00:50:26.478747 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 829/944] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:50:26 2022] GET /compute/v2.1/servers/bfbf8641-41b8-47d8-af4d-ef17c6bd1935 => generated 1358 bytes in 243 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 19 00:50:26.599277 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [req-b5fa393b-a41b-4eba-871e-f9f242d53556 req-6eb3e562-50c7-4fa1-8f39-541e36632c10 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "bfbf8641-41b8-47d8-af4d-ef17c6bd1935", "tag": "aa62f9af-6447-4516-a59f-dc36fd82da11"}]} {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 19 00:50:26.617323 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-b5fa393b-a41b-4eba-871e-f9f242d53556 req-6eb3e562-50c7-4fa1-8f39-541e36632c10 service nova] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:50:26.618070 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-b5fa393b-a41b-4eba-871e-f9f242d53556 req-6eb3e562-50c7-4fa1-8f39-541e36632c10 service nova] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:50:26.618945 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-b5fa393b-a41b-4eba-871e-f9f242d53556 req-6eb3e562-50c7-4fa1-8f39-541e36632c10 service nova] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:50:26.634207 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.compute.server_external_events [req-b5fa393b-a41b-4eba-871e-f9f242d53556 req-6eb3e562-50c7-4fa1-8f39-541e36632c10 service nova] Creating event network-changed:aa62f9af-6447-4516-a59f-dc36fd82da11 for instance bfbf8641-41b8-47d8-af4d-ef17c6bd1935 on hv-cinder-maste Oct 19 00:50:26.642320 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [req-b5fa393b-a41b-4eba-871e-f9f242d53556 req-6eb3e562-50c7-4fa1-8f39-541e36632c10 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.051703 Oct 19 00:50:26.643661 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 830/945] 10.222.0.26 () {64 vars in 1317 bytes} [Wed Oct 19 00:50:26 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 19 00:50:27.518405 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-a4391462-300f-42f9-8539-1832064cff79 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:50:27.519184 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-a4391462-300f-42f9-8539-1832064cff79 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] [instance: bfbf8641-41b8-47d8-af4d-ef17c6bd1935] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:50:27.525859 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-a4391462-300f-42f9-8539-1832064cff79 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:50:27.526348 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-a4391462-300f-42f9-8539-1832064cff79 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:50:27.526817 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-a4391462-300f-42f9-8539-1832064cff79 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:50:27.527537 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-a4391462-300f-42f9-8539-1832064cff79 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:50:27.527993 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-a4391462-300f-42f9-8539-1832064cff79 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:50:27.528422 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-a4391462-300f-42f9-8539-1832064cff79 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:50:27.688446 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-a4391462-300f-42f9-8539-1832064cff79 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '531efd120096452caf8c0b7a7f6b9ba1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a8a6d900d9d344a694b955f51fa46aa9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:50:27.696914 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-a4391462-300f-42f9-8539-1832064cff79 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] 10.222.0.26 "GET /compute/v2.1/servers/bfbf8641-41b8-47d8-af4d-ef17c6bd1935" status: 200 len: 1358 microversion: 2.1 time: 0.183438 Oct 19 00:50:27.697518 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 831/946] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:50:27 2022] GET /compute/v2.1/servers/bfbf8641-41b8-47d8-af4d-ef17c6bd1935 => generated 1358 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 19 00:50:28.721693 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-37fcfa41-876c-46b7-8c84-067cbf5e7cea tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:50:28.722831 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-37fcfa41-876c-46b7-8c84-067cbf5e7cea tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] [instance: bfbf8641-41b8-47d8-af4d-ef17c6bd1935] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:50:28.730076 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-37fcfa41-876c-46b7-8c84-067cbf5e7cea tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:50:28.730663 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-37fcfa41-876c-46b7-8c84-067cbf5e7cea tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:50:28.731146 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-37fcfa41-876c-46b7-8c84-067cbf5e7cea tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:50:28.732789 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-37fcfa41-876c-46b7-8c84-067cbf5e7cea tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:50:28.733884 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-37fcfa41-876c-46b7-8c84-067cbf5e7cea tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:50:28.734367 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-37fcfa41-876c-46b7-8c84-067cbf5e7cea tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:50:28.924012 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-37fcfa41-876c-46b7-8c84-067cbf5e7cea tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '531efd120096452caf8c0b7a7f6b9ba1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a8a6d900d9d344a694b955f51fa46aa9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:50:28.932553 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-37fcfa41-876c-46b7-8c84-067cbf5e7cea tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] 10.222.0.26 "GET /compute/v2.1/servers/bfbf8641-41b8-47d8-af4d-ef17c6bd1935" status: 200 len: 1358 microversion: 2.1 time: 0.216203 Oct 19 00:50:28.933454 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 832/947] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:50:28 2022] GET /compute/v2.1/servers/bfbf8641-41b8-47d8-af4d-ef17c6bd1935 => generated 1358 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 19 00:50:29.957062 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-9027e221-63b0-4281-8791-5f6b16a65f10 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:50:29.958061 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-9027e221-63b0-4281-8791-5f6b16a65f10 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] [instance: bfbf8641-41b8-47d8-af4d-ef17c6bd1935] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:50:29.965573 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-9027e221-63b0-4281-8791-5f6b16a65f10 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:50:29.966185 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-9027e221-63b0-4281-8791-5f6b16a65f10 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:50:29.967439 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-9027e221-63b0-4281-8791-5f6b16a65f10 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:50:29.968263 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-9027e221-63b0-4281-8791-5f6b16a65f10 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:50:29.969040 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-9027e221-63b0-4281-8791-5f6b16a65f10 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:50:29.969508 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-9027e221-63b0-4281-8791-5f6b16a65f10 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:50:30.146588 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-9027e221-63b0-4281-8791-5f6b16a65f10 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '531efd120096452caf8c0b7a7f6b9ba1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a8a6d900d9d344a694b955f51fa46aa9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:50:30.161560 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-9027e221-63b0-4281-8791-5f6b16a65f10 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] 10.222.0.26 "GET /compute/v2.1/servers/bfbf8641-41b8-47d8-af4d-ef17c6bd1935" status: 200 len: 1358 microversion: 2.1 time: 0.207923 Oct 19 00:50:30.162893 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 833/948] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:50:29 2022] GET /compute/v2.1/servers/bfbf8641-41b8-47d8-af4d-ef17c6bd1935 => generated 1358 bytes in 210 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 19 00:50:31.187942 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-2ca1acd8-32ec-4897-8dd3-2bae8f2bddd4 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:50:31.188659 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-2ca1acd8-32ec-4897-8dd3-2bae8f2bddd4 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] [instance: bfbf8641-41b8-47d8-af4d-ef17c6bd1935] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:50:31.195858 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-2ca1acd8-32ec-4897-8dd3-2bae8f2bddd4 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:50:31.196401 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-2ca1acd8-32ec-4897-8dd3-2bae8f2bddd4 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:50:31.196890 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-2ca1acd8-32ec-4897-8dd3-2bae8f2bddd4 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:50:31.197708 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-2ca1acd8-32ec-4897-8dd3-2bae8f2bddd4 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:50:31.198305 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-2ca1acd8-32ec-4897-8dd3-2bae8f2bddd4 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:50:31.198820 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-2ca1acd8-32ec-4897-8dd3-2bae8f2bddd4 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:50:31.350831 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-2ca1acd8-32ec-4897-8dd3-2bae8f2bddd4 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '531efd120096452caf8c0b7a7f6b9ba1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a8a6d900d9d344a694b955f51fa46aa9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:50:31.359393 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-2ca1acd8-32ec-4897-8dd3-2bae8f2bddd4 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] 10.222.0.26 "GET /compute/v2.1/servers/bfbf8641-41b8-47d8-af4d-ef17c6bd1935" status: 200 len: 1358 microversion: 2.1 time: 0.175776 Oct 19 00:50:31.360081 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 834/949] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:50:31 2022] GET /compute/v2.1/servers/bfbf8641-41b8-47d8-af4d-ef17c6bd1935 => generated 1358 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 19 00:50:31.934161 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [req-e81a7317-bd04-4d29-9f0f-0736ee9dc46f req-0bfed1ec-0057-4331-91f2-ab7156ff4395 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "bfbf8641-41b8-47d8-af4d-ef17c6bd1935", "name": "network-vif-plugged", "status": "completed", "tag": "aa62f9af-6447-4516-a59f-dc36fd82da11"}]} {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 19 00:50:31.945499 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-e81a7317-bd04-4d29-9f0f-0736ee9dc46f req-0bfed1ec-0057-4331-91f2-ab7156ff4395 service nova] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:50:31.945997 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-e81a7317-bd04-4d29-9f0f-0736ee9dc46f req-0bfed1ec-0057-4331-91f2-ab7156ff4395 service nova] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:50:31.946437 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-e81a7317-bd04-4d29-9f0f-0736ee9dc46f req-0bfed1ec-0057-4331-91f2-ab7156ff4395 service nova] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:50:31.959810 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.compute.server_external_events [req-e81a7317-bd04-4d29-9f0f-0736ee9dc46f req-0bfed1ec-0057-4331-91f2-ab7156ff4395 service nova] Creating event network-vif-plugged:aa62f9af-6447-4516-a59f-dc36fd82da11 for instance bfbf8641-41b8-47d8-af4d-ef17c6bd1935 on hv-cinder-maste Oct 19 00:50:31.967871 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [req-e81a7317-bd04-4d29-9f0f-0736ee9dc46f req-0bfed1ec-0057-4331-91f2-ab7156ff4395 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.036963 Oct 19 00:50:31.968539 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 835/950] 10.222.0.26 () {64 vars in 1317 bytes} [Wed Oct 19 00:50:31 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 40 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 19 00:50:32.385125 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-113825be-2324-4ddc-b099-3ae24b806f1b tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:50:32.386292 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-113825be-2324-4ddc-b099-3ae24b806f1b tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] [instance: bfbf8641-41b8-47d8-af4d-ef17c6bd1935] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:50:32.394119 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-113825be-2324-4ddc-b099-3ae24b806f1b tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:50:32.394656 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-113825be-2324-4ddc-b099-3ae24b806f1b tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:50:32.395096 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-113825be-2324-4ddc-b099-3ae24b806f1b tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:50:32.395815 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-113825be-2324-4ddc-b099-3ae24b806f1b tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:50:32.396269 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-113825be-2324-4ddc-b099-3ae24b806f1b tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:50:32.396690 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-113825be-2324-4ddc-b099-3ae24b806f1b tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:50:32.550621 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-113825be-2324-4ddc-b099-3ae24b806f1b tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '531efd120096452caf8c0b7a7f6b9ba1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a8a6d900d9d344a694b955f51fa46aa9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:50:32.561126 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-113825be-2324-4ddc-b099-3ae24b806f1b tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] 10.222.0.26 "GET /compute/v2.1/servers/bfbf8641-41b8-47d8-af4d-ef17c6bd1935" status: 200 len: 1358 microversion: 2.1 time: 0.180271 Oct 19 00:50:32.561993 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 836/951] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:50:32 2022] GET /compute/v2.1/servers/bfbf8641-41b8-47d8-af4d-ef17c6bd1935 => generated 1358 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 19 00:50:33.586515 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-abe1495b-fc73-458b-aee3-40b049fc915d tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:50:33.587254 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-abe1495b-fc73-458b-aee3-40b049fc915d tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] [instance: bfbf8641-41b8-47d8-af4d-ef17c6bd1935] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:50:33.594755 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-abe1495b-fc73-458b-aee3-40b049fc915d tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:50:33.595240 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-abe1495b-fc73-458b-aee3-40b049fc915d tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:50:33.595678 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-abe1495b-fc73-458b-aee3-40b049fc915d tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:50:33.596390 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-abe1495b-fc73-458b-aee3-40b049fc915d tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:50:33.596843 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-abe1495b-fc73-458b-aee3-40b049fc915d tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:50:33.597268 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-abe1495b-fc73-458b-aee3-40b049fc915d tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:50:33.743753 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-abe1495b-fc73-458b-aee3-40b049fc915d tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '531efd120096452caf8c0b7a7f6b9ba1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a8a6d900d9d344a694b955f51fa46aa9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:50:33.753341 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-abe1495b-fc73-458b-aee3-40b049fc915d tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] 10.222.0.26 "GET /compute/v2.1/servers/bfbf8641-41b8-47d8-af4d-ef17c6bd1935" status: 200 len: 1358 microversion: 2.1 time: 0.170894 Oct 19 00:50:33.753950 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 837/952] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:50:33 2022] GET /compute/v2.1/servers/bfbf8641-41b8-47d8-af4d-ef17c6bd1935 => generated 1358 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 19 00:50:34.776377 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-cfcaed9f-76d4-4e86-9419-3cec5a3958ff tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:50:34.777105 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-cfcaed9f-76d4-4e86-9419-3cec5a3958ff tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] [instance: bfbf8641-41b8-47d8-af4d-ef17c6bd1935] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:50:34.784360 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-cfcaed9f-76d4-4e86-9419-3cec5a3958ff tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:50:34.784882 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-cfcaed9f-76d4-4e86-9419-3cec5a3958ff tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:50:34.785303 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-cfcaed9f-76d4-4e86-9419-3cec5a3958ff tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:50:34.786148 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-cfcaed9f-76d4-4e86-9419-3cec5a3958ff tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:50:34.786652 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-cfcaed9f-76d4-4e86-9419-3cec5a3958ff tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:50:34.787113 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-cfcaed9f-76d4-4e86-9419-3cec5a3958ff tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:50:34.947842 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-cfcaed9f-76d4-4e86-9419-3cec5a3958ff tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '531efd120096452caf8c0b7a7f6b9ba1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a8a6d900d9d344a694b955f51fa46aa9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:50:34.956998 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-cfcaed9f-76d4-4e86-9419-3cec5a3958ff tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] 10.222.0.26 "GET /compute/v2.1/servers/bfbf8641-41b8-47d8-af4d-ef17c6bd1935" status: 200 len: 1548 microversion: 2.1 time: 0.184546 Oct 19 00:50:34.957797 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 838/953] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:50:34 2022] GET /compute/v2.1/servers/bfbf8641-41b8-47d8-af4d-ef17c6bd1935 => generated 1548 bytes in 186 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 19 00:50:34.984476 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-0888d6be-b3ed-404f-95e5-4ad14916ec14 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:50:34.985089 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-0888d6be-b3ed-404f-95e5-4ad14916ec14 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] [instance: bfbf8641-41b8-47d8-af4d-ef17c6bd1935] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:50:34.991374 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-0888d6be-b3ed-404f-95e5-4ad14916ec14 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:50:34.991880 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-0888d6be-b3ed-404f-95e5-4ad14916ec14 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:50:34.992327 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-0888d6be-b3ed-404f-95e5-4ad14916ec14 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:50:34.993315 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-0888d6be-b3ed-404f-95e5-4ad14916ec14 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:50:34.993802 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-0888d6be-b3ed-404f-95e5-4ad14916ec14 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:50:34.994309 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-0888d6be-b3ed-404f-95e5-4ad14916ec14 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:50:35.174905 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-0888d6be-b3ed-404f-95e5-4ad14916ec14 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '531efd120096452caf8c0b7a7f6b9ba1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a8a6d900d9d344a694b955f51fa46aa9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:50:35.185888 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-0888d6be-b3ed-404f-95e5-4ad14916ec14 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] 10.222.0.26 "GET /compute/v2.1/servers/bfbf8641-41b8-47d8-af4d-ef17c6bd1935" status: 200 len: 1548 microversion: 2.1 time: 0.206000 Oct 19 00:50:35.187172 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 839/954] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:50:34 2022] GET /compute/v2.1/servers/bfbf8641-41b8-47d8-af4d-ef17c6bd1935 => generated 1548 bytes in 208 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 19 00:50:35.217495 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-8303edb6-0cd5-497a-b392-0a587020d7c2 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:50:35.223170 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-8303edb6-0cd5-497a-b392-0a587020d7c2 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Policy check for os_compute_api:servers:allow_all_filters failed with credentials {'is_admin': False, 'user_id': '531efd120096452caf8c0b7a7f6b9ba1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a8a6d900d9d344a694b955f51fa46aa9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:50:35.223994 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-8303edb6-0cd5-497a-b392-0a587020d7c2 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Searching by: {'deleted': False, 'project_id': 'a8a6d900d9d344a694b955f51fa46aa9'} {{(pid=125641) get_all /opt/stack/nova/nova/compute/api.py:3017}} Oct 19 00:50:35.232023 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-8303edb6-0cd5-497a-b392-0a587020d7c2 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:50:35.232663 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-8303edb6-0cd5-497a-b392-0a587020d7c2 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:50:35.233108 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-8303edb6-0cd5-497a-b392-0a587020d7c2 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:50:35.233860 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-8303edb6-0cd5-497a-b392-0a587020d7c2 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:50:35.234387 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-8303edb6-0cd5-497a-b392-0a587020d7c2 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:50:35.234832 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-8303edb6-0cd5-497a-b392-0a587020d7c2 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:50:35.333516 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.multi_cell_list [None req-8303edb6-0cd5-497a-b392-0a587020d7c2 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Listed batch of 1 results from cell out of 1000 limit. Returned 1 total so far. {{(pid=125641) do_query /opt/stack/nova/nova/compute/multi_cell_list.py:373}} Oct 19 00:50:35.387644 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG dbcounter [-] [125641] Writing DB stats nova_cell1:SELECT=100 {{(pid=125641) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 00:50:35.440028 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-8303edb6-0cd5-497a-b392-0a587020d7c2 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] 10.222.0.26 "GET /compute/v2.1/servers" status: 200 len: 343 microversion: 2.1 time: 0.227269 Oct 19 00:50:35.440539 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 840/955] 10.222.0.26 () {58 vars in 1125 bytes} [Wed Oct 19 00:50:35 2022] GET /compute/v2.1/servers => generated 343 bytes in 229 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 19 00:50:35.453254 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-2a49e7c4-1881-4499-9048-2c116ceda7e6 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:50:35.453870 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-2a49e7c4-1881-4499-9048-2c116ceda7e6 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] [instance: bfbf8641-41b8-47d8-af4d-ef17c6bd1935] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:50:35.460255 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-2a49e7c4-1881-4499-9048-2c116ceda7e6 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:50:35.460745 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-2a49e7c4-1881-4499-9048-2c116ceda7e6 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:50:35.461186 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-2a49e7c4-1881-4499-9048-2c116ceda7e6 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:50:35.461905 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-2a49e7c4-1881-4499-9048-2c116ceda7e6 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:50:35.462372 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-2a49e7c4-1881-4499-9048-2c116ceda7e6 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:50:35.462842 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-2a49e7c4-1881-4499-9048-2c116ceda7e6 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:50:35.632291 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-2a49e7c4-1881-4499-9048-2c116ceda7e6 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '531efd120096452caf8c0b7a7f6b9ba1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a8a6d900d9d344a694b955f51fa46aa9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:50:35.640728 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-2a49e7c4-1881-4499-9048-2c116ceda7e6 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] 10.222.0.26 "GET /compute/v2.1/servers/bfbf8641-41b8-47d8-af4d-ef17c6bd1935" status: 200 len: 1548 microversion: 2.1 time: 0.192122 Oct 19 00:50:35.641362 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 841/956] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:50:35 2022] GET /compute/v2.1/servers/bfbf8641-41b8-47d8-af4d-ef17c6bd1935 => generated 1548 bytes in 193 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 19 00:50:37.399931 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-5ea2696e-1248-4283-a762-7b1b0a1f79c2 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "b9d30ae7-badc-4163-827a-140033caedf8"}} {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 19 00:50:37.402231 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-5ea2696e-1248-4283-a762-7b1b0a1f79c2 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] [instance: bfbf8641-41b8-47d8-af4d-ef17c6bd1935] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:50:37.409724 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-5ea2696e-1248-4283-a762-7b1b0a1f79c2 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:50:37.410206 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-5ea2696e-1248-4283-a762-7b1b0a1f79c2 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:50:37.410675 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-5ea2696e-1248-4283-a762-7b1b0a1f79c2 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:50:37.411389 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-5ea2696e-1248-4283-a762-7b1b0a1f79c2 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:50:37.411957 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-5ea2696e-1248-4283-a762-7b1b0a1f79c2 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:50:37.412392 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-5ea2696e-1248-4283-a762-7b1b0a1f79c2 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:50:37.773656 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-5ea2696e-1248-4283-a762-7b1b0a1f79c2 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] 10.222.0.26 "POST /compute/v2.1/servers/bfbf8641-41b8-47d8-af4d-ef17c6bd1935/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.377888 Oct 19 00:50:37.774316 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 842/957] 10.222.0.26 () {60 vars in 1323 bytes} [Wed Oct 19 00:50:37 2022] POST /compute/v2.1/servers/bfbf8641-41b8-47d8-af4d-ef17c6bd1935/os-volume_attachments => generated 194 bytes in 380 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 19 00:50:40.677283 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Oct 19 00:50:40.678229 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: Wed Oct 19 00:50:40 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers (ip 10.222.0.26) !!! Oct 19 00:50:42.844818 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-fc571041-6f55-4652-aaf8-ba465fd4c52b tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:50:42.845676 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-fc571041-6f55-4652-aaf8-ba465fd4c52b tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] [instance: d49a602c-229d-48e6-8b26-c67f73ed8f2f] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:50:42.852135 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-fc571041-6f55-4652-aaf8-ba465fd4c52b tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:50:42.852634 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-fc571041-6f55-4652-aaf8-ba465fd4c52b tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:50:42.853077 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-fc571041-6f55-4652-aaf8-ba465fd4c52b tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:50:42.853816 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-fc571041-6f55-4652-aaf8-ba465fd4c52b tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:50:42.854275 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-fc571041-6f55-4652-aaf8-ba465fd4c52b tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:50:42.854746 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-fc571041-6f55-4652-aaf8-ba465fd4c52b tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:50:42.882068 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-fc571041-6f55-4652-aaf8-ba465fd4c52b tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] 10.222.0.26 "GET /compute/v2.1/servers/d49a602c-229d-48e6-8b26-c67f73ed8f2f/ips" status: 200 len: 145 microversion: 2.1 time: 0.042953 Oct 19 00:50:42.882706 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 843/958] 10.222.0.26 () {58 vars in 1248 bytes} [Wed Oct 19 00:50:42 2022] GET /compute/v2.1/servers/d49a602c-229d-48e6-8b26-c67f73ed8f2f/ips => generated 145 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 19 00:50:44.378958 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-f352d85d-c28f-447a-a729-8049120b6c8a tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "10edff48-5874-4a29-adfb-d0a62bdeca7c"}} {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 19 00:50:44.378958 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-f352d85d-c28f-447a-a729-8049120b6c8a tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] [instance: d49a602c-229d-48e6-8b26-c67f73ed8f2f] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:50:44.385660 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-f352d85d-c28f-447a-a729-8049120b6c8a tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:50:44.386154 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-f352d85d-c28f-447a-a729-8049120b6c8a tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:50:44.386603 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-f352d85d-c28f-447a-a729-8049120b6c8a tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:50:44.387342 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-f352d85d-c28f-447a-a729-8049120b6c8a tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:50:44.388431 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-f352d85d-c28f-447a-a729-8049120b6c8a tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:50:44.389178 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-f352d85d-c28f-447a-a729-8049120b6c8a tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:50:44.940431 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-f352d85d-c28f-447a-a729-8049120b6c8a tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] 10.222.0.26 "POST /compute/v2.1/servers/d49a602c-229d-48e6-8b26-c67f73ed8f2f/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.572044 Oct 19 00:50:44.941326 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 844/959] 10.222.0.26 () {60 vars in 1323 bytes} [Wed Oct 19 00:50:44 2022] POST /compute/v2.1/servers/d49a602c-229d-48e6-8b26-c67f73ed8f2f/os-volume_attachments => generated 194 bytes in 574 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 19 00:50:45.628733 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG dbcounter [-] [125641] Writing DB stats nova_cell0:SELECT=2 {{(pid=125641) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 00:50:45.633258 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-e001781f-1ee6-42c6-bb57-12ad8c7f4968 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:50:45.634132 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-e001781f-1ee6-42c6-bb57-12ad8c7f4968 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] [instance: bfbf8641-41b8-47d8-af4d-ef17c6bd1935] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:50:45.641580 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-e001781f-1ee6-42c6-bb57-12ad8c7f4968 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:50:45.642055 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-e001781f-1ee6-42c6-bb57-12ad8c7f4968 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:50:45.642752 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-e001781f-1ee6-42c6-bb57-12ad8c7f4968 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:50:45.643560 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-e001781f-1ee6-42c6-bb57-12ad8c7f4968 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:50:45.644022 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-e001781f-1ee6-42c6-bb57-12ad8c7f4968 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:50:45.644445 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-e001781f-1ee6-42c6-bb57-12ad8c7f4968 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:50:45.858407 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-e001781f-1ee6-42c6-bb57-12ad8c7f4968 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '531efd120096452caf8c0b7a7f6b9ba1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a8a6d900d9d344a694b955f51fa46aa9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:50:45.886701 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-e001781f-1ee6-42c6-bb57-12ad8c7f4968 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] 10.222.0.26 "GET /compute/v2.1/servers/bfbf8641-41b8-47d8-af4d-ef17c6bd1935" status: 200 len: 1594 microversion: 2.1 time: 0.259389 Oct 19 00:50:45.887768 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 845/960] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:50:45 2022] GET /compute/v2.1/servers/bfbf8641-41b8-47d8-af4d-ef17c6bd1935 => generated 1594 bytes in 261 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 19 00:50:47.428947 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Oct 19 00:50:47.429892 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: Wed Oct 19 00:50:47 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers (ip 10.222.0.26) !!! Oct 19 00:50:48.027706 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [req-1ee1c058-7c99-484e-b2eb-7717fa710bb8 req-75b85b30-517d-4222-a521-3de52ede615a service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "bfbf8641-41b8-47d8-af4d-ef17c6bd1935", "tag": "aa62f9af-6447-4516-a59f-dc36fd82da11"}]} {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 19 00:50:48.038713 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-1ee1c058-7c99-484e-b2eb-7717fa710bb8 req-75b85b30-517d-4222-a521-3de52ede615a service nova] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:50:48.039200 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-1ee1c058-7c99-484e-b2eb-7717fa710bb8 req-75b85b30-517d-4222-a521-3de52ede615a service nova] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:50:48.039762 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-1ee1c058-7c99-484e-b2eb-7717fa710bb8 req-75b85b30-517d-4222-a521-3de52ede615a service nova] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:50:48.052747 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.compute.server_external_events [req-1ee1c058-7c99-484e-b2eb-7717fa710bb8 req-75b85b30-517d-4222-a521-3de52ede615a service nova] Creating event network-changed:aa62f9af-6447-4516-a59f-dc36fd82da11 for instance bfbf8641-41b8-47d8-af4d-ef17c6bd1935 on hv-cinder-maste Oct 19 00:50:48.058238 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [req-1ee1c058-7c99-484e-b2eb-7717fa710bb8 req-75b85b30-517d-4222-a521-3de52ede615a service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.034163 Oct 19 00:50:48.058888 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 846/961] 10.222.0.26 () {64 vars in 1317 bytes} [Wed Oct 19 00:50:48 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 19 00:50:49.481611 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-eb77070c-a932-48eb-9a5e-f6b06dc28eb6 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Action: 'action', calling method: >, body: {"addSecurityGroup": {"name": "tempest-secgroup-smoke-1982202811"}} {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 19 00:50:49.482962 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-eb77070c-a932-48eb-9a5e-f6b06dc28eb6 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] [instance: bfbf8641-41b8-47d8-af4d-ef17c6bd1935] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:50:49.491616 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-eb77070c-a932-48eb-9a5e-f6b06dc28eb6 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:50:49.492292 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-eb77070c-a932-48eb-9a5e-f6b06dc28eb6 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:50:49.492742 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-eb77070c-a932-48eb-9a5e-f6b06dc28eb6 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:50:49.493473 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-eb77070c-a932-48eb-9a5e-f6b06dc28eb6 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:50:49.493933 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-eb77070c-a932-48eb-9a5e-f6b06dc28eb6 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:50:49.494361 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-eb77070c-a932-48eb-9a5e-f6b06dc28eb6 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:50:49.651122 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.network.security_group_api [None req-eb77070c-a932-48eb-9a5e-f6b06dc28eb6 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Adding security group 871fe458-7962-462f-9d9c-b229a52f3fc4 to port aa62f9af-6447-4516-a59f-dc36fd82da11 Oct 19 00:50:50.102340 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-eb77070c-a932-48eb-9a5e-f6b06dc28eb6 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] 10.222.0.26 "POST /compute/v2.1/servers/bfbf8641-41b8-47d8-af4d-ef17c6bd1935/action" status: 202 len: 0 microversion: 2.1 time: 0.630415 Oct 19 00:50:50.107249 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 847/962] 10.222.0.26 () {60 vars in 1278 bytes} [Wed Oct 19 00:50:49 2022] POST /compute/v2.1/servers/bfbf8641-41b8-47d8-af4d-ef17c6bd1935/action => generated 0 bytes in 633 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 19 00:50:50.113663 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [req-0463b068-2d48-414d-98ab-a1eba1322e79 req-c14312ea-349a-4c4e-bf27-6f7bda839942 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "bfbf8641-41b8-47d8-af4d-ef17c6bd1935", "tag": "aa62f9af-6447-4516-a59f-dc36fd82da11"}]} {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 19 00:50:50.118956 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG dbcounter [-] [125641] Writing DB stats nova_api:SELECT=48 {{(pid=125641) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 00:50:50.132644 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-0463b068-2d48-414d-98ab-a1eba1322e79 req-c14312ea-349a-4c4e-bf27-6f7bda839942 service nova] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:50:50.133115 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-0463b068-2d48-414d-98ab-a1eba1322e79 req-c14312ea-349a-4c4e-bf27-6f7bda839942 service nova] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:50:50.133775 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-0463b068-2d48-414d-98ab-a1eba1322e79 req-c14312ea-349a-4c4e-bf27-6f7bda839942 service nova] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:50:50.148013 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.compute.server_external_events [req-0463b068-2d48-414d-98ab-a1eba1322e79 req-c14312ea-349a-4c4e-bf27-6f7bda839942 service nova] Creating event network-changed:aa62f9af-6447-4516-a59f-dc36fd82da11 for instance bfbf8641-41b8-47d8-af4d-ef17c6bd1935 on hv-cinder-maste Oct 19 00:50:50.155220 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [req-0463b068-2d48-414d-98ab-a1eba1322e79 req-c14312ea-349a-4c4e-bf27-6f7bda839942 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.046511 Oct 19 00:50:50.156154 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 848/963] 10.222.0.26 () {64 vars in 1317 bytes} [Wed Oct 19 00:50:50 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 19 00:50:50.162442 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-5cb7fbca-144d-4988-a9ea-acc9f9421b9e tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:50:50.163361 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-5cb7fbca-144d-4988-a9ea-acc9f9421b9e tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] [instance: bfbf8641-41b8-47d8-af4d-ef17c6bd1935] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:50:50.170332 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-5cb7fbca-144d-4988-a9ea-acc9f9421b9e tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:50:50.170861 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-5cb7fbca-144d-4988-a9ea-acc9f9421b9e tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:50:50.171299 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-5cb7fbca-144d-4988-a9ea-acc9f9421b9e tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:50:50.172023 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-5cb7fbca-144d-4988-a9ea-acc9f9421b9e tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:50:50.172468 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-5cb7fbca-144d-4988-a9ea-acc9f9421b9e tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:50:50.172924 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-5cb7fbca-144d-4988-a9ea-acc9f9421b9e tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:50:50.423305 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-5cb7fbca-144d-4988-a9ea-acc9f9421b9e tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '531efd120096452caf8c0b7a7f6b9ba1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a8a6d900d9d344a694b955f51fa46aa9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:50:50.441209 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-5cb7fbca-144d-4988-a9ea-acc9f9421b9e tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] 10.222.0.26 "GET /compute/v2.1/servers/bfbf8641-41b8-47d8-af4d-ef17c6bd1935" status: 200 len: 1760 microversion: 2.1 time: 0.283547 Oct 19 00:50:50.442114 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 849/964] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:50:50 2022] GET /compute/v2.1/servers/bfbf8641-41b8-47d8-af4d-ef17c6bd1935 => generated 1760 bytes in 285 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:50:52.737723 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-e261b599-a690-43c3-8e37-c08d9a3c7598 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:50:52.740155 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-e261b599-a690-43c3-8e37-c08d9a3c7598 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] [instance: d49a602c-229d-48e6-8b26-c67f73ed8f2f] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:50:52.748950 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-e261b599-a690-43c3-8e37-c08d9a3c7598 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:50:52.749488 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-e261b599-a690-43c3-8e37-c08d9a3c7598 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:50:52.749961 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-e261b599-a690-43c3-8e37-c08d9a3c7598 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:50:52.750771 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-e261b599-a690-43c3-8e37-c08d9a3c7598 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:50:52.751273 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-e261b599-a690-43c3-8e37-c08d9a3c7598 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:50:52.751737 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-e261b599-a690-43c3-8e37-c08d9a3c7598 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:50:52.786629 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-e261b599-a690-43c3-8e37-c08d9a3c7598 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] 10.222.0.26 "GET /compute/v2.1/servers/d49a602c-229d-48e6-8b26-c67f73ed8f2f/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.053112 Oct 19 00:50:52.787297 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 850/965] 10.222.0.26 () {58 vars in 1302 bytes} [Wed Oct 19 00:50:52 2022] GET /compute/v2.1/servers/d49a602c-229d-48e6-8b26-c67f73ed8f2f/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 19 00:50:52.799005 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-5d013d5b-c1d9-4d69-9518-c9bc39ad68a2 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:50:52.799501 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-5d013d5b-c1d9-4d69-9518-c9bc39ad68a2 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] [instance: d49a602c-229d-48e6-8b26-c67f73ed8f2f] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:50:52.805938 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-5d013d5b-c1d9-4d69-9518-c9bc39ad68a2 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:50:52.806601 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-5d013d5b-c1d9-4d69-9518-c9bc39ad68a2 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:50:52.807146 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-5d013d5b-c1d9-4d69-9518-c9bc39ad68a2 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:50:52.808053 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-5d013d5b-c1d9-4d69-9518-c9bc39ad68a2 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:50:52.808556 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-5d013d5b-c1d9-4d69-9518-c9bc39ad68a2 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:50:52.809656 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-5d013d5b-c1d9-4d69-9518-c9bc39ad68a2 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:50:52.843891 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-5d013d5b-c1d9-4d69-9518-c9bc39ad68a2 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] 10.222.0.26 "GET /compute/v2.1/servers/d49a602c-229d-48e6-8b26-c67f73ed8f2f/os-volume_attachments/10edff48-5874-4a29-adfb-d0a62bdeca7c" status: 200 len: 194 microversion: 2.1 time: 0.048628 Oct 19 00:50:52.844520 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 851/966] 10.222.0.26 () {58 vars in 1413 bytes} [Wed Oct 19 00:50:52 2022] GET /compute/v2.1/servers/d49a602c-229d-48e6-8b26-c67f73ed8f2f/os-volume_attachments/10edff48-5874-4a29-adfb-d0a62bdeca7c => generated 194 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 19 00:50:54.252643 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-08a32fda-7b0f-4f35-9708-9f0bd3a17cbf tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "d7bfb291-38e8-43f1-9b9b-650c487c4c89"}} {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 19 00:50:54.255057 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-08a32fda-7b0f-4f35-9708-9f0bd3a17cbf tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] [instance: d49a602c-229d-48e6-8b26-c67f73ed8f2f] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:50:54.261963 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-08a32fda-7b0f-4f35-9708-9f0bd3a17cbf tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:50:54.263025 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-08a32fda-7b0f-4f35-9708-9f0bd3a17cbf tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:50:54.263497 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-08a32fda-7b0f-4f35-9708-9f0bd3a17cbf tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:50:54.264223 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-08a32fda-7b0f-4f35-9708-9f0bd3a17cbf tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:50:54.264714 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-08a32fda-7b0f-4f35-9708-9f0bd3a17cbf tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:50:54.265112 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-08a32fda-7b0f-4f35-9708-9f0bd3a17cbf tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:50:54.599322 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-08a32fda-7b0f-4f35-9708-9f0bd3a17cbf tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] 10.222.0.26 "POST /compute/v2.1/servers/d49a602c-229d-48e6-8b26-c67f73ed8f2f/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.350438 Oct 19 00:50:54.606469 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 852/967] 10.222.0.26 () {60 vars in 1323 bytes} [Wed Oct 19 00:50:54 2022] POST /compute/v2.1/servers/d49a602c-229d-48e6-8b26-c67f73ed8f2f/os-volume_attachments => generated 194 bytes in 353 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 19 00:51:02.221719 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-a0983e33-a934-4707-a407-b10f5adbb608 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:51:02.224073 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-a0983e33-a934-4707-a407-b10f5adbb608 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] [instance: d49a602c-229d-48e6-8b26-c67f73ed8f2f] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:51:02.230993 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-a0983e33-a934-4707-a407-b10f5adbb608 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:51:02.231487 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-a0983e33-a934-4707-a407-b10f5adbb608 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:51:02.231951 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-a0983e33-a934-4707-a407-b10f5adbb608 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:51:02.232682 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-a0983e33-a934-4707-a407-b10f5adbb608 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:51:02.233149 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-a0983e33-a934-4707-a407-b10f5adbb608 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:51:02.233569 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-a0983e33-a934-4707-a407-b10f5adbb608 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:51:02.269038 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-a0983e33-a934-4707-a407-b10f5adbb608 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] 10.222.0.26 "GET /compute/v2.1/servers/d49a602c-229d-48e6-8b26-c67f73ed8f2f/os-volume_attachments" status: 200 len: 371 microversion: 2.1 time: 0.052768 Oct 19 00:51:02.269635 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 853/968] 10.222.0.26 () {58 vars in 1302 bytes} [Wed Oct 19 00:51:02 2022] GET /compute/v2.1/servers/d49a602c-229d-48e6-8b26-c67f73ed8f2f/os-volume_attachments => generated 371 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 19 00:51:02.417621 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-38035d49-e5f9-41c7-8d2c-e5235e7dafcc tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:51:02.418465 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-38035d49-e5f9-41c7-8d2c-e5235e7dafcc tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] [instance: d49a602c-229d-48e6-8b26-c67f73ed8f2f] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:51:02.425369 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-38035d49-e5f9-41c7-8d2c-e5235e7dafcc tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:51:02.425852 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-38035d49-e5f9-41c7-8d2c-e5235e7dafcc tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:51:02.426279 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-38035d49-e5f9-41c7-8d2c-e5235e7dafcc tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:51:02.427015 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-38035d49-e5f9-41c7-8d2c-e5235e7dafcc tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:51:02.427465 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-38035d49-e5f9-41c7-8d2c-e5235e7dafcc tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:51:02.427886 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-38035d49-e5f9-41c7-8d2c-e5235e7dafcc tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:51:02.463854 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-38035d49-e5f9-41c7-8d2c-e5235e7dafcc tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] 10.222.0.26 "GET /compute/v2.1/servers/d49a602c-229d-48e6-8b26-c67f73ed8f2f/os-volume_attachments/10edff48-5874-4a29-adfb-d0a62bdeca7c" status: 200 len: 194 microversion: 2.1 time: 0.051008 Oct 19 00:51:02.464491 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 854/969] 10.222.0.26 () {58 vars in 1413 bytes} [Wed Oct 19 00:51:02 2022] GET /compute/v2.1/servers/d49a602c-229d-48e6-8b26-c67f73ed8f2f/os-volume_attachments/10edff48-5874-4a29-adfb-d0a62bdeca7c => generated 194 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 19 00:51:02.476527 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-8af4d944-4b81-47f5-b919-743cddc75829 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:51:02.477041 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-8af4d944-4b81-47f5-b919-743cddc75829 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] [instance: d49a602c-229d-48e6-8b26-c67f73ed8f2f] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:51:02.484452 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-8af4d944-4b81-47f5-b919-743cddc75829 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:51:02.485024 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-8af4d944-4b81-47f5-b919-743cddc75829 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:51:02.485527 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-8af4d944-4b81-47f5-b919-743cddc75829 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:51:02.486357 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-8af4d944-4b81-47f5-b919-743cddc75829 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:51:02.486924 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-8af4d944-4b81-47f5-b919-743cddc75829 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:51:02.487348 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-8af4d944-4b81-47f5-b919-743cddc75829 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:51:02.492559 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG dbcounter [-] [125641] Writing DB stats nova_cell1:SELECT=91,nova_cell1:UPDATE=6,nova_cell1:INSERT=3 {{(pid=125641) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 00:51:02.590058 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.objects.instance [None req-8af4d944-4b81-47f5-b919-743cddc75829 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lazy-loading 'services' on Instance uuid d49a602c-229d-48e6-8b26-c67f73ed8f2f {{(pid=125641) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 19 00:51:02.679307 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-8af4d944-4b81-47f5-b919-743cddc75829 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] 10.222.0.26 "DELETE /compute/v2.1/servers/d49a602c-229d-48e6-8b26-c67f73ed8f2f/os-volume_attachments/10edff48-5874-4a29-adfb-d0a62bdeca7c" status: 202 len: 0 microversion: 2.1 time: 0.207130 Oct 19 00:51:02.679823 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 855/970] 10.222.0.26 () {58 vars in 1416 bytes} [Wed Oct 19 00:51:02 2022] DELETE /compute/v2.1/servers/d49a602c-229d-48e6-8b26-c67f73ed8f2f/os-volume_attachments/10edff48-5874-4a29-adfb-d0a62bdeca7c => generated 0 bytes in 209 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Oct 19 00:51:03.165457 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Oct 19 00:51:03.166171 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: Wed Oct 19 00:51:03 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers (ip 10.222.0.26) !!! Oct 19 00:51:04.891354 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-32094e52-2cec-4382-b31e-729fef060229 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:51:04.892006 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-32094e52-2cec-4382-b31e-729fef060229 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] [instance: d49a602c-229d-48e6-8b26-c67f73ed8f2f] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:51:04.899528 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-32094e52-2cec-4382-b31e-729fef060229 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:51:04.900199 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-32094e52-2cec-4382-b31e-729fef060229 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:51:04.900799 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-32094e52-2cec-4382-b31e-729fef060229 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:51:04.901590 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-32094e52-2cec-4382-b31e-729fef060229 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:51:04.902123 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-32094e52-2cec-4382-b31e-729fef060229 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:51:04.902637 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-32094e52-2cec-4382-b31e-729fef060229 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:51:04.936426 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-32094e52-2cec-4382-b31e-729fef060229 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] 10.222.0.26 "GET /compute/v2.1/servers/d49a602c-229d-48e6-8b26-c67f73ed8f2f/os-volume_attachments/d7bfb291-38e8-43f1-9b9b-650c487c4c89" status: 200 len: 194 microversion: 2.1 time: 0.050345 Oct 19 00:51:04.937048 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 856/971] 10.222.0.26 () {58 vars in 1413 bytes} [Wed Oct 19 00:51:04 2022] GET /compute/v2.1/servers/d49a602c-229d-48e6-8b26-c67f73ed8f2f/os-volume_attachments/d7bfb291-38e8-43f1-9b9b-650c487c4c89 => generated 194 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 19 00:51:04.952944 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-56aa8995-c610-4022-8b07-441c2f98cdbd tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:51:04.953435 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-56aa8995-c610-4022-8b07-441c2f98cdbd tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] [instance: d49a602c-229d-48e6-8b26-c67f73ed8f2f] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:51:04.960196 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-56aa8995-c610-4022-8b07-441c2f98cdbd tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:51:04.960686 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-56aa8995-c610-4022-8b07-441c2f98cdbd tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:51:04.961128 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-56aa8995-c610-4022-8b07-441c2f98cdbd tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:51:04.961844 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-56aa8995-c610-4022-8b07-441c2f98cdbd tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:51:04.962297 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-56aa8995-c610-4022-8b07-441c2f98cdbd tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:51:04.962760 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-56aa8995-c610-4022-8b07-441c2f98cdbd tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:51:05.078000 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.objects.instance [None req-56aa8995-c610-4022-8b07-441c2f98cdbd tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lazy-loading 'services' on Instance uuid d49a602c-229d-48e6-8b26-c67f73ed8f2f {{(pid=125641) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 19 00:51:05.163740 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-56aa8995-c610-4022-8b07-441c2f98cdbd tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] 10.222.0.26 "DELETE /compute/v2.1/servers/d49a602c-229d-48e6-8b26-c67f73ed8f2f/os-volume_attachments/d7bfb291-38e8-43f1-9b9b-650c487c4c89" status: 202 len: 0 microversion: 2.1 time: 0.215104 Oct 19 00:51:05.164778 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 857/972] 10.222.0.26 () {58 vars in 1416 bytes} [Wed Oct 19 00:51:04 2022] DELETE /compute/v2.1/servers/d49a602c-229d-48e6-8b26-c67f73ed8f2f/os-volume_attachments/d7bfb291-38e8-43f1-9b9b-650c487c4c89 => generated 0 bytes in 218 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Oct 19 00:51:06.429037 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-0274d58f-4f1f-4d17-be7e-43e99f6585cb tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:51:06.431421 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-0274d58f-4f1f-4d17-be7e-43e99f6585cb tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] [instance: d49a602c-229d-48e6-8b26-c67f73ed8f2f] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:51:06.438961 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-0274d58f-4f1f-4d17-be7e-43e99f6585cb tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:51:06.439447 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-0274d58f-4f1f-4d17-be7e-43e99f6585cb tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:51:06.439884 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-0274d58f-4f1f-4d17-be7e-43e99f6585cb tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:51:06.440814 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-0274d58f-4f1f-4d17-be7e-43e99f6585cb tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:51:06.441274 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-0274d58f-4f1f-4d17-be7e-43e99f6585cb tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:51:06.441697 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-0274d58f-4f1f-4d17-be7e-43e99f6585cb tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:51:06.476387 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-0274d58f-4f1f-4d17-be7e-43e99f6585cb tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] 10.222.0.26 "GET /compute/v2.1/servers/d49a602c-229d-48e6-8b26-c67f73ed8f2f/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.052330 Oct 19 00:51:06.476977 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 858/973] 10.222.0.26 () {58 vars in 1302 bytes} [Wed Oct 19 00:51:06 2022] GET /compute/v2.1/servers/d49a602c-229d-48e6-8b26-c67f73ed8f2f/os-volume_attachments => generated 25 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 356 bytes (2 switches on core 0) Oct 19 00:51:06.650159 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-4cbaae69-dc3b-4541-9a85-13401db05dd4 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:51:06.652731 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-4cbaae69-dc3b-4541-9a85-13401db05dd4 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] [instance: d49a602c-229d-48e6-8b26-c67f73ed8f2f] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:51:06.659336 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-4cbaae69-dc3b-4541-9a85-13401db05dd4 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:51:06.659800 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-4cbaae69-dc3b-4541-9a85-13401db05dd4 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:51:06.660242 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-4cbaae69-dc3b-4541-9a85-13401db05dd4 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:51:06.660949 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-4cbaae69-dc3b-4541-9a85-13401db05dd4 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:51:06.661406 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-4cbaae69-dc3b-4541-9a85-13401db05dd4 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:51:06.661829 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-4cbaae69-dc3b-4541-9a85-13401db05dd4 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:51:06.695017 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-4cbaae69-dc3b-4541-9a85-13401db05dd4 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] 10.222.0.26 "GET /compute/v2.1/servers/d49a602c-229d-48e6-8b26-c67f73ed8f2f/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.048788 Oct 19 00:51:06.695655 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 859/974] 10.222.0.26 () {58 vars in 1302 bytes} [Wed Oct 19 00:51:06 2022] GET /compute/v2.1/servers/d49a602c-229d-48e6-8b26-c67f73ed8f2f/os-volume_attachments => generated 25 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 356 bytes (2 switches on core 0) Oct 19 00:51:06.770422 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-5c3ca8d3-e01b-48bf-88a2-ea9fd0ce2f9e tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:51:06.771315 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-5c3ca8d3-e01b-48bf-88a2-ea9fd0ce2f9e tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] [instance: d49a602c-229d-48e6-8b26-c67f73ed8f2f] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:51:06.783217 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-5c3ca8d3-e01b-48bf-88a2-ea9fd0ce2f9e tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:51:06.784063 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-5c3ca8d3-e01b-48bf-88a2-ea9fd0ce2f9e tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:51:06.784832 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-5c3ca8d3-e01b-48bf-88a2-ea9fd0ce2f9e tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:51:06.786073 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-5c3ca8d3-e01b-48bf-88a2-ea9fd0ce2f9e tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:51:06.786894 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-5c3ca8d3-e01b-48bf-88a2-ea9fd0ce2f9e tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:51:06.787611 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-5c3ca8d3-e01b-48bf-88a2-ea9fd0ce2f9e tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:51:06.828046 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-5c3ca8d3-e01b-48bf-88a2-ea9fd0ce2f9e tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] [instance: d49a602c-229d-48e6-8b26-c67f73ed8f2f] Going to try to terminate instance {{(pid=125641) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 19 00:51:06.922990 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-5c3ca8d3-e01b-48bf-88a2-ea9fd0ce2f9e tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] 10.222.0.26 "DELETE /compute/v2.1/servers/d49a602c-229d-48e6-8b26-c67f73ed8f2f" status: 204 len: 0 microversion: 2.1 time: 0.158967 Oct 19 00:51:06.923948 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 860/975] 10.222.0.26 () {58 vars in 1239 bytes} [Wed Oct 19 00:51:06 2022] DELETE /compute/v2.1/servers/d49a602c-229d-48e6-8b26-c67f73ed8f2f => generated 0 bytes in 162 msecs (HTTP/1.1 204) 9 headers in 363 bytes (2 switches on core 0) Oct 19 00:51:06.935888 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-069b281f-14ff-4fb2-94f1-e4278ee8f91f tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:51:06.936661 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-069b281f-14ff-4fb2-94f1-e4278ee8f91f tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] [instance: d49a602c-229d-48e6-8b26-c67f73ed8f2f] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:51:06.943301 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-069b281f-14ff-4fb2-94f1-e4278ee8f91f tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:51:06.943764 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-069b281f-14ff-4fb2-94f1-e4278ee8f91f tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:51:06.944192 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-069b281f-14ff-4fb2-94f1-e4278ee8f91f tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:51:06.944908 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-069b281f-14ff-4fb2-94f1-e4278ee8f91f tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:51:06.945361 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-069b281f-14ff-4fb2-94f1-e4278ee8f91f tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:51:06.945777 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-069b281f-14ff-4fb2-94f1-e4278ee8f91f tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:51:07.117198 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-069b281f-14ff-4fb2-94f1-e4278ee8f91f tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a48ed2d9ded741ca91afc1fbbd9c608f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '82d7723ef95b4c3faeac9835eeb233a5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:51:07.125554 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-069b281f-14ff-4fb2-94f1-e4278ee8f91f tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] 10.222.0.26 "GET /compute/v2.1/servers/d49a602c-229d-48e6-8b26-c67f73ed8f2f" status: 200 len: 1674 microversion: 2.1 time: 0.195085 Oct 19 00:51:07.126217 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 861/976] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:51:06 2022] GET /compute/v2.1/servers/d49a602c-229d-48e6-8b26-c67f73ed8f2f => generated 1674 bytes in 197 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 19 00:51:08.149596 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-a25d8c71-b35c-49c6-b9c9-18d3732848c0 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:51:08.150512 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-a25d8c71-b35c-49c6-b9c9-18d3732848c0 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] [instance: d49a602c-229d-48e6-8b26-c67f73ed8f2f] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:51:08.158227 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-a25d8c71-b35c-49c6-b9c9-18d3732848c0 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:51:08.158754 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-a25d8c71-b35c-49c6-b9c9-18d3732848c0 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:51:08.159188 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-a25d8c71-b35c-49c6-b9c9-18d3732848c0 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:51:08.159895 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-a25d8c71-b35c-49c6-b9c9-18d3732848c0 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:51:08.160345 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-a25d8c71-b35c-49c6-b9c9-18d3732848c0 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:51:08.160766 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-a25d8c71-b35c-49c6-b9c9-18d3732848c0 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:51:08.353080 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-a25d8c71-b35c-49c6-b9c9-18d3732848c0 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a48ed2d9ded741ca91afc1fbbd9c608f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '82d7723ef95b4c3faeac9835eeb233a5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:51:08.361943 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-a25d8c71-b35c-49c6-b9c9-18d3732848c0 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] 10.222.0.26 "GET /compute/v2.1/servers/d49a602c-229d-48e6-8b26-c67f73ed8f2f" status: 200 len: 1674 microversion: 2.1 time: 0.216286 Oct 19 00:51:08.362627 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 862/977] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:51:08 2022] GET /compute/v2.1/servers/d49a602c-229d-48e6-8b26-c67f73ed8f2f => generated 1674 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 19 00:51:09.393082 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-6c62a405-f504-4c7e-a377-d97d68b091fb tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:51:09.394104 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-6c62a405-f504-4c7e-a377-d97d68b091fb tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] [instance: d49a602c-229d-48e6-8b26-c67f73ed8f2f] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:51:09.402177 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-6c62a405-f504-4c7e-a377-d97d68b091fb tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:51:09.402745 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-6c62a405-f504-4c7e-a377-d97d68b091fb tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:51:09.403167 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-6c62a405-f504-4c7e-a377-d97d68b091fb tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:51:09.403899 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-6c62a405-f504-4c7e-a377-d97d68b091fb tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:51:09.404359 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-6c62a405-f504-4c7e-a377-d97d68b091fb tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:51:09.404789 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-6c62a405-f504-4c7e-a377-d97d68b091fb tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:51:09.588062 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-6c62a405-f504-4c7e-a377-d97d68b091fb tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a48ed2d9ded741ca91afc1fbbd9c608f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '82d7723ef95b4c3faeac9835eeb233a5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:51:09.598124 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-6c62a405-f504-4c7e-a377-d97d68b091fb tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] 10.222.0.26 "GET /compute/v2.1/servers/d49a602c-229d-48e6-8b26-c67f73ed8f2f" status: 200 len: 1674 microversion: 2.1 time: 0.212857 Oct 19 00:51:09.599000 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 863/978] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:51:09 2022] GET /compute/v2.1/servers/d49a602c-229d-48e6-8b26-c67f73ed8f2f => generated 1674 bytes in 215 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 19 00:51:10.263084 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [req-68521ad8-9047-440e-b314-6bada176eba5 req-f42f642f-618e-42d0-90bb-23b77f0db001 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "d49a602c-229d-48e6-8b26-c67f73ed8f2f", "name": "network-vif-deleted", "tag": "a6830382-e592-4ea8-9d6a-63da173d5d01"}]} {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 19 00:51:10.291443 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-68521ad8-9047-440e-b314-6bada176eba5 req-f42f642f-618e-42d0-90bb-23b77f0db001 service nova] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:51:10.292504 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-68521ad8-9047-440e-b314-6bada176eba5 req-f42f642f-618e-42d0-90bb-23b77f0db001 service nova] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:51:10.293943 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-68521ad8-9047-440e-b314-6bada176eba5 req-f42f642f-618e-42d0-90bb-23b77f0db001 service nova] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:51:10.323547 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.compute.server_external_events [req-68521ad8-9047-440e-b314-6bada176eba5 req-f42f642f-618e-42d0-90bb-23b77f0db001 service nova] Creating event network-vif-deleted:a6830382-e592-4ea8-9d6a-63da173d5d01 for instance d49a602c-229d-48e6-8b26-c67f73ed8f2f on hv-cinder-maste Oct 19 00:51:10.332135 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [req-68521ad8-9047-440e-b314-6bada176eba5 req-f42f642f-618e-42d0-90bb-23b77f0db001 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.208349 Oct 19 00:51:10.333045 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 864/979] 10.222.0.26 () {64 vars in 1317 bytes} [Wed Oct 19 00:51:10 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 211 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 19 00:51:10.628186 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-b53572aa-304c-4f82-b406-1eeab0ef0c8f tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:51:10.629483 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-b53572aa-304c-4f82-b406-1eeab0ef0c8f tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] [instance: d49a602c-229d-48e6-8b26-c67f73ed8f2f] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:51:10.642180 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-b53572aa-304c-4f82-b406-1eeab0ef0c8f tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:51:10.643090 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-b53572aa-304c-4f82-b406-1eeab0ef0c8f tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:51:10.643857 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-b53572aa-304c-4f82-b406-1eeab0ef0c8f tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:51:10.645064 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-b53572aa-304c-4f82-b406-1eeab0ef0c8f tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:51:10.645893 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-b53572aa-304c-4f82-b406-1eeab0ef0c8f tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:51:10.646741 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-b53572aa-304c-4f82-b406-1eeab0ef0c8f tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:51:10.690355 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-b53572aa-304c-4f82-b406-1eeab0ef0c8f tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:51:10.690922 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-b53572aa-304c-4f82-b406-1eeab0ef0c8f tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:51:10.691407 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-b53572aa-304c-4f82-b406-1eeab0ef0c8f tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:51:10.691928 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.objects.instance [None req-b53572aa-304c-4f82-b406-1eeab0ef0c8f tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lazy-loading 'fault' on Instance uuid d49a602c-229d-48e6-8b26-c67f73ed8f2f {{(pid=125641) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 19 00:51:10.752824 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-b53572aa-304c-4f82-b406-1eeab0ef0c8f tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a48ed2d9ded741ca91afc1fbbd9c608f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '82d7723ef95b4c3faeac9835eeb233a5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:51:10.762652 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-b53572aa-304c-4f82-b406-1eeab0ef0c8f tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] 10.222.0.26 "GET /compute/v2.1/servers/d49a602c-229d-48e6-8b26-c67f73ed8f2f" status: 200 len: 1328 microversion: 2.1 time: 0.142692 Oct 19 00:51:10.763348 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 865/980] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:51:10 2022] GET /compute/v2.1/servers/d49a602c-229d-48e6-8b26-c67f73ed8f2f => generated 1328 bytes in 145 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:51:11.799790 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-12c58e93-4fde-4e72-ad48-d0b83e5be671 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:51:11.800514 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-12c58e93-4fde-4e72-ad48-d0b83e5be671 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] [instance: d49a602c-229d-48e6-8b26-c67f73ed8f2f] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:51:11.807340 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-12c58e93-4fde-4e72-ad48-d0b83e5be671 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:51:11.807829 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-12c58e93-4fde-4e72-ad48-d0b83e5be671 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:51:11.808268 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-12c58e93-4fde-4e72-ad48-d0b83e5be671 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:51:11.809008 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-12c58e93-4fde-4e72-ad48-d0b83e5be671 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:51:11.809468 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-12c58e93-4fde-4e72-ad48-d0b83e5be671 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:51:11.809899 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-12c58e93-4fde-4e72-ad48-d0b83e5be671 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:51:11.825439 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.wsgi [None req-12c58e93-4fde-4e72-ad48-d0b83e5be671 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] HTTP exception thrown: Instance d49a602c-229d-48e6-8b26-c67f73ed8f2f could not be found. Oct 19 00:51:11.827188 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-12c58e93-4fde-4e72-ad48-d0b83e5be671 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Request method failure captured: Oct 19 00:51:11.827188 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: request: GET /compute/v2.1/servers/d49a602c-229d-48e6-8b26-c67f73ed8f2f HTTP/1.1 Oct 19 00:51:11.827188 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Accept: application/json Oct 19 00:51:11.827188 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Accept-Encoding: identity Oct 19 00:51:11.827188 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Connection: close Oct 19 00:51:11.827188 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Content-Length: 0 Oct 19 00:51:11.827188 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Content-Type: application/json Oct 19 00:51:11.827188 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Host: 10.222.0.26 Oct 19 00:51:11.827188 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Openstack-System-Scope: None Oct 19 00:51:11.827188 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: User-Agent: python-urllib3/1.26.12 Oct 19 00:51:11.827188 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Auth-Token: gAAAAABjT0jftXzQfZyc8CRHQV4W_gddvzuI97tl8zjpIOHAUUDNwJcgCLYc0G45lhVAYuRkTnFQ_q2IREDQvezXcz0pdRx04PeX9oCthNL6COthZ_7lYmlxWElgdQkmAYCIVWlggqLn29lN1bm77huNlmbpHl2KvOIdUnCRzvDMG4IjBMuh1Jc Oct 19 00:51:11.827188 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Domain-Id: None Oct 19 00:51:11.827188 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Domain-Name: None Oct 19 00:51:11.827188 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Identity-Status: Confirmed Oct 19 00:51:11.827188 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Is-Admin-Project: True Oct 19 00:51:11.827188 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Project-Domain-Id: default Oct 19 00:51:11.827188 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Project-Domain-Name: Default Oct 19 00:51:11.827188 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Project-Id: 82d7723ef95b4c3faeac9835eeb233a5 Oct 19 00:51:11.827188 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Project-Name: tempest-AttachVolumeTestJSON-35550062 Oct 19 00:51:11.827188 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Role: member,reader Oct 19 00:51:11.827188 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Roles: member,reader Oct 19 00:51:11.830225 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/82d7723ef95b4c3faeac9835eeb233a5"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/82d7723ef95b4c3faeac9835eeb233a5"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/82d7723ef95b4c3faeac9835eeb233a5"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_82d7723ef95b4c3faeac9835eeb233a5"}]}] Oct 19 00:51:11.830225 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Tenant: tempest-AttachVolumeTestJSON-35550062 Oct 19 00:51:11.830225 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Tenant-Id: 82d7723ef95b4c3faeac9835eeb233a5 Oct 19 00:51:11.830225 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Tenant-Name: tempest-AttachVolumeTestJSON-35550062 Oct 19 00:51:11.830225 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User: tempest-AttachVolumeTestJSON-35550062-project Oct 19 00:51:11.830225 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User-Domain-Id: default Oct 19 00:51:11.830225 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User-Domain-Name: Default Oct 19 00:51:11.830225 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User-Id: a48ed2d9ded741ca91afc1fbbd9c608f Oct 19 00:51:11.830225 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User-Name: tempest-AttachVolumeTestJSON-35550062-project Oct 19 00:51:11.830225 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: method: > Oct 19 00:51:11.830225 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: exception: Instance d49a602c-229d-48e6-8b26-c67f73ed8f2f could not be found. Oct 19 00:51:11.830225 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: action_args: {'id': 'd49a602c-229d-48e6-8b26-c67f73ed8f2f'} Oct 19 00:51:11.831936 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 19 00:51:11.831936 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 19 00:51:11.831936 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 19 00:51:11.831936 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 19 00:51:11.831936 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 19 00:51:11.831936 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 19 00:51:11.831936 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 19 00:51:11.831936 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 19 00:51:11.831936 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 19 00:51:11.831936 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 19 00:51:11.831936 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance d49a602c-229d-48e6-8b26-c67f73ed8f2f could not be found. Oct 19 00:51:11.831936 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:51:11.831936 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 19 00:51:11.831936 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:51:11.831936 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 19 00:51:11.831936 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 19 00:51:11.831936 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 19 00:51:11.831936 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 19 00:51:11.831936 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 19 00:51:11.831936 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 19 00:51:11.835412 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 19 00:51:11.835412 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 19 00:51:11.835412 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 19 00:51:11.835412 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 19 00:51:11.835412 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 19 00:51:11.835412 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 19 00:51:11.835412 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 19 00:51:11.835412 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance d49a602c-229d-48e6-8b26-c67f73ed8f2f could not be found. Oct 19 00:51:11.835412 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:51:11.835412 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 19 00:51:11.835412 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:51:11.835412 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 19 00:51:11.835412 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 19 00:51:11.835412 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 19 00:51:11.835412 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 19 00:51:11.835412 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 19 00:51:11.835412 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance d49a602c-229d-48e6-8b26-c67f73ed8f2f could not be found. Oct 19 00:51:11.835412 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:51:11.837915 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-12c58e93-4fde-4e72-ad48-d0b83e5be671 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Returning 404 to user: Instance d49a602c-229d-48e6-8b26-c67f73ed8f2f could not be found. {{(pid=125641) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 19 00:51:11.837915 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-12c58e93-4fde-4e72-ad48-d0b83e5be671 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] 10.222.0.26 "GET /compute/v2.1/servers/d49a602c-229d-48e6-8b26-c67f73ed8f2f" status: 404 len: 111 microversion: 2.1 time: 0.032920 Oct 19 00:51:11.837915 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 866/981] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:51:11 2022] GET /compute/v2.1/servers/d49a602c-229d-48e6-8b26-c67f73ed8f2f => generated 111 bytes in 34 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 19 00:51:11.840937 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-3ef5e2ec-2ff4-4f81-9655-6c2e42213c7e tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Calling method '.version_select at 0x7f71c3926550>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:51:11.862934 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-3ef5e2ec-2ff4-4f81-9655-6c2e42213c7e tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] 10.222.0.26 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-2113229266" status: 202 len: 0 microversion: 2.1 time: 0.027527 Oct 19 00:51:11.863723 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 867/982] 10.222.0.26 () {58 vars in 1221 bytes} [Wed Oct 19 00:51:11 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-2113229266 => generated 0 bytes in 30 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Oct 19 00:51:13.072336 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-8ff2be29-2ed0-4f0d-a1cf-6bef01e0ddb7 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Action: 'action', calling method: >, body: {"reboot": {"type": "SOFT"}} {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 19 00:51:13.074773 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-8ff2be29-2ed0-4f0d-a1cf-6bef01e0ddb7 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] [instance: bfbf8641-41b8-47d8-af4d-ef17c6bd1935] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:51:13.082111 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-8ff2be29-2ed0-4f0d-a1cf-6bef01e0ddb7 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:51:13.082633 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-8ff2be29-2ed0-4f0d-a1cf-6bef01e0ddb7 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:51:13.083059 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-8ff2be29-2ed0-4f0d-a1cf-6bef01e0ddb7 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:51:13.083759 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-8ff2be29-2ed0-4f0d-a1cf-6bef01e0ddb7 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:51:13.084213 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-8ff2be29-2ed0-4f0d-a1cf-6bef01e0ddb7 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:51:13.084643 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-8ff2be29-2ed0-4f0d-a1cf-6bef01e0ddb7 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:51:13.181783 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-8ff2be29-2ed0-4f0d-a1cf-6bef01e0ddb7 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] 10.222.0.26 "POST /compute/v2.1/servers/bfbf8641-41b8-47d8-af4d-ef17c6bd1935/action" status: 202 len: 0 microversion: 2.1 time: 0.113627 Oct 19 00:51:13.182944 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 868/983] 10.222.0.26 () {60 vars in 1278 bytes} [Wed Oct 19 00:51:13 2022] POST /compute/v2.1/servers/bfbf8641-41b8-47d8-af4d-ef17c6bd1935/action => generated 0 bytes in 116 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Oct 19 00:51:13.194804 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-e9546395-2ca6-486a-8384-226814e56203 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:51:13.196037 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-e9546395-2ca6-486a-8384-226814e56203 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] [instance: bfbf8641-41b8-47d8-af4d-ef17c6bd1935] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:51:13.207140 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-e9546395-2ca6-486a-8384-226814e56203 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:51:13.207744 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-e9546395-2ca6-486a-8384-226814e56203 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:51:13.208440 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-e9546395-2ca6-486a-8384-226814e56203 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:51:13.209694 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-e9546395-2ca6-486a-8384-226814e56203 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:51:13.210529 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-e9546395-2ca6-486a-8384-226814e56203 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:51:13.211292 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-e9546395-2ca6-486a-8384-226814e56203 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:51:13.414518 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-e9546395-2ca6-486a-8384-226814e56203 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '531efd120096452caf8c0b7a7f6b9ba1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a8a6d900d9d344a694b955f51fa46aa9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:51:13.422293 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG dbcounter [-] [125641] Writing DB stats nova_cell1:SELECT=94,nova_cell1:INSERT=4,nova_cell1:UPDATE=2 {{(pid=125641) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 00:51:13.435515 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-e9546395-2ca6-486a-8384-226814e56203 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] 10.222.0.26 "GET /compute/v2.1/servers/bfbf8641-41b8-47d8-af4d-ef17c6bd1935" status: 200 len: 1752 microversion: 2.1 time: 0.246768 Oct 19 00:51:13.436892 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 869/984] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:51:13 2022] GET /compute/v2.1/servers/bfbf8641-41b8-47d8-af4d-ef17c6bd1935 => generated 1752 bytes in 249 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 19 00:51:14.462545 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-6ffe3709-e62c-41a4-8f62-63841f5515ce tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:51:14.463247 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-6ffe3709-e62c-41a4-8f62-63841f5515ce tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] [instance: bfbf8641-41b8-47d8-af4d-ef17c6bd1935] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:51:14.472677 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-6ffe3709-e62c-41a4-8f62-63841f5515ce tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:51:14.473080 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-6ffe3709-e62c-41a4-8f62-63841f5515ce tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:51:14.473521 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-6ffe3709-e62c-41a4-8f62-63841f5515ce tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:51:14.474242 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-6ffe3709-e62c-41a4-8f62-63841f5515ce tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:51:14.474725 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-6ffe3709-e62c-41a4-8f62-63841f5515ce tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:51:14.475170 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-6ffe3709-e62c-41a4-8f62-63841f5515ce tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:51:14.675155 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-6ffe3709-e62c-41a4-8f62-63841f5515ce tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '531efd120096452caf8c0b7a7f6b9ba1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a8a6d900d9d344a694b955f51fa46aa9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:51:14.685027 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-6ffe3709-e62c-41a4-8f62-63841f5515ce tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] 10.222.0.26 "GET /compute/v2.1/servers/bfbf8641-41b8-47d8-af4d-ef17c6bd1935" status: 200 len: 1757 microversion: 2.1 time: 0.226939 Oct 19 00:51:14.685629 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 870/985] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:51:14 2022] GET /compute/v2.1/servers/bfbf8641-41b8-47d8-af4d-ef17c6bd1935 => generated 1757 bytes in 229 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:51:14.970053 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-0b26b474-ed84-4377-96f4-af71eb80e9d8 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:51:14.970687 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-0b26b474-ed84-4377-96f4-af71eb80e9d8 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] [instance: d49a602c-229d-48e6-8b26-c67f73ed8f2f] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:51:14.977395 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-0b26b474-ed84-4377-96f4-af71eb80e9d8 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:51:14.977884 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-0b26b474-ed84-4377-96f4-af71eb80e9d8 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:51:14.978315 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-0b26b474-ed84-4377-96f4-af71eb80e9d8 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:51:14.979063 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-0b26b474-ed84-4377-96f4-af71eb80e9d8 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:51:14.979515 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-0b26b474-ed84-4377-96f4-af71eb80e9d8 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:51:14.979961 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-0b26b474-ed84-4377-96f4-af71eb80e9d8 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:51:14.995483 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.wsgi [None req-0b26b474-ed84-4377-96f4-af71eb80e9d8 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] HTTP exception thrown: Instance d49a602c-229d-48e6-8b26-c67f73ed8f2f could not be found. Oct 19 00:51:14.998024 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-0b26b474-ed84-4377-96f4-af71eb80e9d8 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Request method failure captured: Oct 19 00:51:14.998024 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: request: DELETE /compute/v2.1/servers/d49a602c-229d-48e6-8b26-c67f73ed8f2f HTTP/1.1 Oct 19 00:51:14.998024 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Accept: application/json Oct 19 00:51:14.998024 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Accept-Encoding: identity Oct 19 00:51:14.998024 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Connection: close Oct 19 00:51:14.998024 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Content-Length: 0 Oct 19 00:51:14.998024 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Content-Type: application/json Oct 19 00:51:14.998024 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Host: 10.222.0.26 Oct 19 00:51:14.998024 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Openstack-System-Scope: None Oct 19 00:51:14.998024 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: User-Agent: python-urllib3/1.26.12 Oct 19 00:51:14.998024 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Auth-Token: gAAAAABjT0jftXzQfZyc8CRHQV4W_gddvzuI97tl8zjpIOHAUUDNwJcgCLYc0G45lhVAYuRkTnFQ_q2IREDQvezXcz0pdRx04PeX9oCthNL6COthZ_7lYmlxWElgdQkmAYCIVWlggqLn29lN1bm77huNlmbpHl2KvOIdUnCRzvDMG4IjBMuh1Jc Oct 19 00:51:14.998024 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Domain-Id: None Oct 19 00:51:14.998024 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Domain-Name: None Oct 19 00:51:14.998024 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Identity-Status: Confirmed Oct 19 00:51:14.998024 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Is-Admin-Project: True Oct 19 00:51:14.998024 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Project-Domain-Id: default Oct 19 00:51:14.998024 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Project-Domain-Name: Default Oct 19 00:51:14.998024 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Project-Id: 82d7723ef95b4c3faeac9835eeb233a5 Oct 19 00:51:14.998024 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Project-Name: tempest-AttachVolumeTestJSON-35550062 Oct 19 00:51:14.998024 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Role: member,reader Oct 19 00:51:14.998024 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Roles: member,reader Oct 19 00:51:15.000013 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/82d7723ef95b4c3faeac9835eeb233a5"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/82d7723ef95b4c3faeac9835eeb233a5"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/82d7723ef95b4c3faeac9835eeb233a5"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_82d7723ef95b4c3faeac9835eeb233a5"}]}] Oct 19 00:51:15.000013 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Tenant: tempest-AttachVolumeTestJSON-35550062 Oct 19 00:51:15.000013 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Tenant-Id: 82d7723ef95b4c3faeac9835eeb233a5 Oct 19 00:51:15.000013 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Tenant-Name: tempest-AttachVolumeTestJSON-35550062 Oct 19 00:51:15.000013 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User: tempest-AttachVolumeTestJSON-35550062-project Oct 19 00:51:15.000013 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User-Domain-Id: default Oct 19 00:51:15.000013 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User-Domain-Name: Default Oct 19 00:51:15.000013 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User-Id: a48ed2d9ded741ca91afc1fbbd9c608f Oct 19 00:51:15.000013 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User-Name: tempest-AttachVolumeTestJSON-35550062-project Oct 19 00:51:15.000013 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: method: > Oct 19 00:51:15.000013 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: exception: Instance d49a602c-229d-48e6-8b26-c67f73ed8f2f could not be found. Oct 19 00:51:15.000013 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: action_args: {'id': 'd49a602c-229d-48e6-8b26-c67f73ed8f2f'} Oct 19 00:51:15.001238 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 19 00:51:15.001238 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 19 00:51:15.001238 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 19 00:51:15.001238 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 19 00:51:15.001238 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 19 00:51:15.001238 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 19 00:51:15.001238 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 19 00:51:15.001238 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 19 00:51:15.001238 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 19 00:51:15.001238 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 19 00:51:15.001238 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance d49a602c-229d-48e6-8b26-c67f73ed8f2f could not be found. Oct 19 00:51:15.001238 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:51:15.001238 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 19 00:51:15.001238 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:51:15.001238 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 19 00:51:15.001238 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 19 00:51:15.001238 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 19 00:51:15.001238 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 19 00:51:15.001238 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 19 00:51:15.001238 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 19 00:51:15.003572 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 19 00:51:15.003572 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Oct 19 00:51:15.003572 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Oct 19 00:51:15.003572 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Oct 19 00:51:15.003572 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Oct 19 00:51:15.003572 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 19 00:51:15.003572 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 19 00:51:15.003572 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 19 00:51:15.003572 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 19 00:51:15.003572 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance d49a602c-229d-48e6-8b26-c67f73ed8f2f could not be found. Oct 19 00:51:15.003572 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:51:15.003572 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 19 00:51:15.003572 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:51:15.003572 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 19 00:51:15.003572 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 19 00:51:15.003572 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 19 00:51:15.003572 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 19 00:51:15.003572 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 19 00:51:15.003572 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance d49a602c-229d-48e6-8b26-c67f73ed8f2f could not be found. Oct 19 00:51:15.003572 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:51:15.006100 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-0b26b474-ed84-4377-96f4-af71eb80e9d8 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Returning 404 to user: Instance d49a602c-229d-48e6-8b26-c67f73ed8f2f could not be found. {{(pid=125641) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 19 00:51:15.006100 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-0b26b474-ed84-4377-96f4-af71eb80e9d8 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] 10.222.0.26 "DELETE /compute/v2.1/servers/d49a602c-229d-48e6-8b26-c67f73ed8f2f" status: 404 len: 111 microversion: 2.1 time: 0.035264 Oct 19 00:51:15.006100 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 871/986] 10.222.0.26 () {58 vars in 1239 bytes} [Wed Oct 19 00:51:14 2022] DELETE /compute/v2.1/servers/d49a602c-229d-48e6-8b26-c67f73ed8f2f => generated 111 bytes in 36 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Oct 19 00:51:15.010508 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-7d35a511-f199-447e-9dc0-c8cfe219e653 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:51:15.011191 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-7d35a511-f199-447e-9dc0-c8cfe219e653 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] [instance: d49a602c-229d-48e6-8b26-c67f73ed8f2f] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:51:15.018783 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-7d35a511-f199-447e-9dc0-c8cfe219e653 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:51:15.019274 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-7d35a511-f199-447e-9dc0-c8cfe219e653 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:51:15.019710 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-7d35a511-f199-447e-9dc0-c8cfe219e653 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:51:15.020429 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-7d35a511-f199-447e-9dc0-c8cfe219e653 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:51:15.020883 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-7d35a511-f199-447e-9dc0-c8cfe219e653 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:51:15.021303 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-7d35a511-f199-447e-9dc0-c8cfe219e653 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:51:15.037048 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.wsgi [None req-7d35a511-f199-447e-9dc0-c8cfe219e653 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] HTTP exception thrown: Instance d49a602c-229d-48e6-8b26-c67f73ed8f2f could not be found. Oct 19 00:51:15.038710 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-7d35a511-f199-447e-9dc0-c8cfe219e653 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Request method failure captured: Oct 19 00:51:15.038710 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: request: GET /compute/v2.1/servers/d49a602c-229d-48e6-8b26-c67f73ed8f2f HTTP/1.1 Oct 19 00:51:15.038710 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Accept: application/json Oct 19 00:51:15.038710 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Accept-Encoding: identity Oct 19 00:51:15.038710 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Connection: close Oct 19 00:51:15.038710 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Content-Length: 0 Oct 19 00:51:15.038710 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Content-Type: application/json Oct 19 00:51:15.038710 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Host: 10.222.0.26 Oct 19 00:51:15.038710 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Openstack-System-Scope: None Oct 19 00:51:15.038710 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: User-Agent: python-urllib3/1.26.12 Oct 19 00:51:15.038710 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Auth-Token: gAAAAABjT0jftXzQfZyc8CRHQV4W_gddvzuI97tl8zjpIOHAUUDNwJcgCLYc0G45lhVAYuRkTnFQ_q2IREDQvezXcz0pdRx04PeX9oCthNL6COthZ_7lYmlxWElgdQkmAYCIVWlggqLn29lN1bm77huNlmbpHl2KvOIdUnCRzvDMG4IjBMuh1Jc Oct 19 00:51:15.038710 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Domain-Id: None Oct 19 00:51:15.038710 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Domain-Name: None Oct 19 00:51:15.038710 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Identity-Status: Confirmed Oct 19 00:51:15.038710 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Is-Admin-Project: True Oct 19 00:51:15.038710 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Project-Domain-Id: default Oct 19 00:51:15.038710 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Project-Domain-Name: Default Oct 19 00:51:15.038710 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Project-Id: 82d7723ef95b4c3faeac9835eeb233a5 Oct 19 00:51:15.038710 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Project-Name: tempest-AttachVolumeTestJSON-35550062 Oct 19 00:51:15.038710 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Role: member,reader Oct 19 00:51:15.038710 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Roles: member,reader Oct 19 00:51:15.041690 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/82d7723ef95b4c3faeac9835eeb233a5"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/82d7723ef95b4c3faeac9835eeb233a5"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/82d7723ef95b4c3faeac9835eeb233a5"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_82d7723ef95b4c3faeac9835eeb233a5"}]}] Oct 19 00:51:15.041690 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Tenant: tempest-AttachVolumeTestJSON-35550062 Oct 19 00:51:15.041690 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Tenant-Id: 82d7723ef95b4c3faeac9835eeb233a5 Oct 19 00:51:15.041690 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Tenant-Name: tempest-AttachVolumeTestJSON-35550062 Oct 19 00:51:15.041690 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User: tempest-AttachVolumeTestJSON-35550062-project Oct 19 00:51:15.041690 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User-Domain-Id: default Oct 19 00:51:15.041690 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User-Domain-Name: Default Oct 19 00:51:15.041690 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User-Id: a48ed2d9ded741ca91afc1fbbd9c608f Oct 19 00:51:15.041690 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User-Name: tempest-AttachVolumeTestJSON-35550062-project Oct 19 00:51:15.041690 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: method: > Oct 19 00:51:15.041690 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: exception: Instance d49a602c-229d-48e6-8b26-c67f73ed8f2f could not be found. Oct 19 00:51:15.041690 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: action_args: {'id': 'd49a602c-229d-48e6-8b26-c67f73ed8f2f'} Oct 19 00:51:15.043429 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 19 00:51:15.043429 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 19 00:51:15.043429 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 19 00:51:15.043429 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 19 00:51:15.043429 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 19 00:51:15.043429 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 19 00:51:15.043429 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 19 00:51:15.043429 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 19 00:51:15.043429 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 19 00:51:15.043429 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 19 00:51:15.043429 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance d49a602c-229d-48e6-8b26-c67f73ed8f2f could not be found. Oct 19 00:51:15.043429 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:51:15.043429 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 19 00:51:15.043429 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:51:15.043429 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 19 00:51:15.043429 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 19 00:51:15.043429 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 19 00:51:15.043429 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 19 00:51:15.043429 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 19 00:51:15.043429 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 19 00:51:15.046242 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 19 00:51:15.046242 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 19 00:51:15.046242 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 19 00:51:15.046242 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 19 00:51:15.046242 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 19 00:51:15.046242 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 19 00:51:15.046242 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 19 00:51:15.046242 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance d49a602c-229d-48e6-8b26-c67f73ed8f2f could not be found. Oct 19 00:51:15.046242 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:51:15.046242 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 19 00:51:15.046242 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:51:15.046242 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 19 00:51:15.046242 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 19 00:51:15.046242 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 19 00:51:15.046242 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 19 00:51:15.046242 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 19 00:51:15.046242 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance d49a602c-229d-48e6-8b26-c67f73ed8f2f could not be found. Oct 19 00:51:15.046242 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:51:15.048799 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-7d35a511-f199-447e-9dc0-c8cfe219e653 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Returning 404 to user: Instance d49a602c-229d-48e6-8b26-c67f73ed8f2f could not be found. {{(pid=125641) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 19 00:51:15.048799 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-7d35a511-f199-447e-9dc0-c8cfe219e653 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] 10.222.0.26 "GET /compute/v2.1/servers/d49a602c-229d-48e6-8b26-c67f73ed8f2f" status: 404 len: 111 microversion: 2.1 time: 0.034208 Oct 19 00:51:15.048799 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 872/987] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:51:15 2022] GET /compute/v2.1/servers/d49a602c-229d-48e6-8b26-c67f73ed8f2f => generated 111 bytes in 36 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 19 00:51:15.711158 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-6be95db8-9051-4924-a3d5-41360897f8a0 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:51:15.712068 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-6be95db8-9051-4924-a3d5-41360897f8a0 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] [instance: bfbf8641-41b8-47d8-af4d-ef17c6bd1935] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:51:15.719190 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-6be95db8-9051-4924-a3d5-41360897f8a0 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:51:15.719725 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-6be95db8-9051-4924-a3d5-41360897f8a0 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:51:15.720170 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-6be95db8-9051-4924-a3d5-41360897f8a0 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:51:15.720892 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-6be95db8-9051-4924-a3d5-41360897f8a0 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:51:15.721349 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-6be95db8-9051-4924-a3d5-41360897f8a0 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:51:15.721778 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-6be95db8-9051-4924-a3d5-41360897f8a0 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:51:15.880566 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-6be95db8-9051-4924-a3d5-41360897f8a0 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '531efd120096452caf8c0b7a7f6b9ba1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a8a6d900d9d344a694b955f51fa46aa9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:51:15.889959 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-6be95db8-9051-4924-a3d5-41360897f8a0 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] 10.222.0.26 "GET /compute/v2.1/servers/bfbf8641-41b8-47d8-af4d-ef17c6bd1935" status: 200 len: 1757 microversion: 2.1 time: 0.183108 Oct 19 00:51:15.890580 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 873/988] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:51:15 2022] GET /compute/v2.1/servers/bfbf8641-41b8-47d8-af4d-ef17c6bd1935 => generated 1757 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 19 00:51:16.213667 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-487a894d-7cdf-4196-a5ed-7da77a7262c2 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:51:16.214258 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-487a894d-7cdf-4196-a5ed-7da77a7262c2 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] [instance: b38d4219-8912-4fd9-9d02-7848016f7954] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:51:16.223681 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-487a894d-7cdf-4196-a5ed-7da77a7262c2 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:51:16.224269 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-487a894d-7cdf-4196-a5ed-7da77a7262c2 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:51:16.224825 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-487a894d-7cdf-4196-a5ed-7da77a7262c2 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:51:16.225774 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-487a894d-7cdf-4196-a5ed-7da77a7262c2 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:51:16.226319 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-487a894d-7cdf-4196-a5ed-7da77a7262c2 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:51:16.226900 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-487a894d-7cdf-4196-a5ed-7da77a7262c2 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:51:16.243661 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.wsgi [None req-487a894d-7cdf-4196-a5ed-7da77a7262c2 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] HTTP exception thrown: Instance b38d4219-8912-4fd9-9d02-7848016f7954 could not be found. Oct 19 00:51:16.245398 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-487a894d-7cdf-4196-a5ed-7da77a7262c2 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Request method failure captured: Oct 19 00:51:16.245398 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: request: DELETE /compute/v2.1/servers/b38d4219-8912-4fd9-9d02-7848016f7954 HTTP/1.1 Oct 19 00:51:16.245398 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Accept: application/json Oct 19 00:51:16.245398 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Accept-Encoding: identity Oct 19 00:51:16.245398 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Connection: close Oct 19 00:51:16.245398 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Content-Length: 0 Oct 19 00:51:16.245398 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Content-Type: application/json Oct 19 00:51:16.245398 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Host: 10.222.0.26 Oct 19 00:51:16.245398 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Openstack-System-Scope: None Oct 19 00:51:16.245398 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: User-Agent: python-urllib3/1.26.12 Oct 19 00:51:16.245398 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Auth-Token: gAAAAABjT0jftXzQfZyc8CRHQV4W_gddvzuI97tl8zjpIOHAUUDNwJcgCLYc0G45lhVAYuRkTnFQ_q2IREDQvezXcz0pdRx04PeX9oCthNL6COthZ_7lYmlxWElgdQkmAYCIVWlggqLn29lN1bm77huNlmbpHl2KvOIdUnCRzvDMG4IjBMuh1Jc Oct 19 00:51:16.245398 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Domain-Id: None Oct 19 00:51:16.245398 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Domain-Name: None Oct 19 00:51:16.245398 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Identity-Status: Confirmed Oct 19 00:51:16.245398 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Is-Admin-Project: True Oct 19 00:51:16.245398 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Project-Domain-Id: default Oct 19 00:51:16.245398 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Project-Domain-Name: Default Oct 19 00:51:16.245398 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Project-Id: 82d7723ef95b4c3faeac9835eeb233a5 Oct 19 00:51:16.245398 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Project-Name: tempest-AttachVolumeTestJSON-35550062 Oct 19 00:51:16.245398 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Role: member,reader Oct 19 00:51:16.245398 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Roles: member,reader Oct 19 00:51:16.247913 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/82d7723ef95b4c3faeac9835eeb233a5"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/82d7723ef95b4c3faeac9835eeb233a5"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/82d7723ef95b4c3faeac9835eeb233a5"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_82d7723ef95b4c3faeac9835eeb233a5"}]}] Oct 19 00:51:16.247913 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Tenant: tempest-AttachVolumeTestJSON-35550062 Oct 19 00:51:16.247913 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Tenant-Id: 82d7723ef95b4c3faeac9835eeb233a5 Oct 19 00:51:16.247913 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Tenant-Name: tempest-AttachVolumeTestJSON-35550062 Oct 19 00:51:16.247913 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User: tempest-AttachVolumeTestJSON-35550062-project Oct 19 00:51:16.247913 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User-Domain-Id: default Oct 19 00:51:16.247913 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User-Domain-Name: Default Oct 19 00:51:16.247913 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User-Id: a48ed2d9ded741ca91afc1fbbd9c608f Oct 19 00:51:16.247913 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User-Name: tempest-AttachVolumeTestJSON-35550062-project Oct 19 00:51:16.247913 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: method: > Oct 19 00:51:16.247913 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: exception: Instance b38d4219-8912-4fd9-9d02-7848016f7954 could not be found. Oct 19 00:51:16.247913 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: action_args: {'id': 'b38d4219-8912-4fd9-9d02-7848016f7954'} Oct 19 00:51:16.249577 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 19 00:51:16.249577 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 19 00:51:16.249577 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 19 00:51:16.249577 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 19 00:51:16.249577 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 19 00:51:16.249577 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 19 00:51:16.249577 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 19 00:51:16.249577 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 19 00:51:16.249577 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 19 00:51:16.249577 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 19 00:51:16.249577 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance b38d4219-8912-4fd9-9d02-7848016f7954 could not be found. Oct 19 00:51:16.249577 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:51:16.249577 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 19 00:51:16.249577 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:51:16.249577 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 19 00:51:16.249577 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 19 00:51:16.249577 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 19 00:51:16.249577 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 19 00:51:16.249577 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 19 00:51:16.249577 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 19 00:51:16.252227 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 19 00:51:16.252227 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Oct 19 00:51:16.252227 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Oct 19 00:51:16.252227 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Oct 19 00:51:16.252227 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Oct 19 00:51:16.252227 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 19 00:51:16.252227 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 19 00:51:16.252227 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 19 00:51:16.252227 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 19 00:51:16.252227 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance b38d4219-8912-4fd9-9d02-7848016f7954 could not be found. Oct 19 00:51:16.252227 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:51:16.252227 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 19 00:51:16.252227 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:51:16.252227 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 19 00:51:16.252227 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 19 00:51:16.252227 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 19 00:51:16.252227 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 19 00:51:16.252227 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 19 00:51:16.252227 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance b38d4219-8912-4fd9-9d02-7848016f7954 could not be found. Oct 19 00:51:16.252227 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:51:16.254915 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-487a894d-7cdf-4196-a5ed-7da77a7262c2 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Returning 404 to user: Instance b38d4219-8912-4fd9-9d02-7848016f7954 could not be found. {{(pid=125641) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 19 00:51:16.254915 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-487a894d-7cdf-4196-a5ed-7da77a7262c2 tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] 10.222.0.26 "DELETE /compute/v2.1/servers/b38d4219-8912-4fd9-9d02-7848016f7954" status: 404 len: 111 microversion: 2.1 time: 0.038288 Oct 19 00:51:16.254915 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 874/989] 10.222.0.26 () {58 vars in 1239 bytes} [Wed Oct 19 00:51:16 2022] DELETE /compute/v2.1/servers/b38d4219-8912-4fd9-9d02-7848016f7954 => generated 111 bytes in 40 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Oct 19 00:51:16.257923 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-019fc280-79eb-4987-9d76-a99160d7dcbe tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:51:16.258577 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-019fc280-79eb-4987-9d76-a99160d7dcbe tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] [instance: b38d4219-8912-4fd9-9d02-7848016f7954] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:51:16.265846 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-019fc280-79eb-4987-9d76-a99160d7dcbe tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:51:16.266339 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-019fc280-79eb-4987-9d76-a99160d7dcbe tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:51:16.266886 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-019fc280-79eb-4987-9d76-a99160d7dcbe tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:51:16.267609 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-019fc280-79eb-4987-9d76-a99160d7dcbe tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:51:16.268060 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-019fc280-79eb-4987-9d76-a99160d7dcbe tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:51:16.268484 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-019fc280-79eb-4987-9d76-a99160d7dcbe tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:51:16.284818 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.wsgi [None req-019fc280-79eb-4987-9d76-a99160d7dcbe tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] HTTP exception thrown: Instance b38d4219-8912-4fd9-9d02-7848016f7954 could not be found. Oct 19 00:51:16.286392 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-019fc280-79eb-4987-9d76-a99160d7dcbe tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Request method failure captured: Oct 19 00:51:16.286392 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: request: GET /compute/v2.1/servers/b38d4219-8912-4fd9-9d02-7848016f7954 HTTP/1.1 Oct 19 00:51:16.286392 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Accept: application/json Oct 19 00:51:16.286392 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Accept-Encoding: identity Oct 19 00:51:16.286392 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Connection: close Oct 19 00:51:16.286392 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Content-Length: 0 Oct 19 00:51:16.286392 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Content-Type: application/json Oct 19 00:51:16.286392 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Host: 10.222.0.26 Oct 19 00:51:16.286392 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Openstack-System-Scope: None Oct 19 00:51:16.286392 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: User-Agent: python-urllib3/1.26.12 Oct 19 00:51:16.286392 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Auth-Token: gAAAAABjT0jftXzQfZyc8CRHQV4W_gddvzuI97tl8zjpIOHAUUDNwJcgCLYc0G45lhVAYuRkTnFQ_q2IREDQvezXcz0pdRx04PeX9oCthNL6COthZ_7lYmlxWElgdQkmAYCIVWlggqLn29lN1bm77huNlmbpHl2KvOIdUnCRzvDMG4IjBMuh1Jc Oct 19 00:51:16.286392 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Domain-Id: None Oct 19 00:51:16.286392 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Domain-Name: None Oct 19 00:51:16.286392 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Identity-Status: Confirmed Oct 19 00:51:16.286392 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Is-Admin-Project: True Oct 19 00:51:16.286392 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Project-Domain-Id: default Oct 19 00:51:16.286392 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Project-Domain-Name: Default Oct 19 00:51:16.286392 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Project-Id: 82d7723ef95b4c3faeac9835eeb233a5 Oct 19 00:51:16.286392 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Project-Name: tempest-AttachVolumeTestJSON-35550062 Oct 19 00:51:16.286392 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Role: member,reader Oct 19 00:51:16.286392 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Roles: member,reader Oct 19 00:51:16.289085 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/82d7723ef95b4c3faeac9835eeb233a5"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/82d7723ef95b4c3faeac9835eeb233a5"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/82d7723ef95b4c3faeac9835eeb233a5"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_82d7723ef95b4c3faeac9835eeb233a5"}]}] Oct 19 00:51:16.289085 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Tenant: tempest-AttachVolumeTestJSON-35550062 Oct 19 00:51:16.289085 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Tenant-Id: 82d7723ef95b4c3faeac9835eeb233a5 Oct 19 00:51:16.289085 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Tenant-Name: tempest-AttachVolumeTestJSON-35550062 Oct 19 00:51:16.289085 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User: tempest-AttachVolumeTestJSON-35550062-project Oct 19 00:51:16.289085 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User-Domain-Id: default Oct 19 00:51:16.289085 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User-Domain-Name: Default Oct 19 00:51:16.289085 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User-Id: a48ed2d9ded741ca91afc1fbbd9c608f Oct 19 00:51:16.289085 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User-Name: tempest-AttachVolumeTestJSON-35550062-project Oct 19 00:51:16.289085 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: method: > Oct 19 00:51:16.289085 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: exception: Instance b38d4219-8912-4fd9-9d02-7848016f7954 could not be found. Oct 19 00:51:16.289085 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: action_args: {'id': 'b38d4219-8912-4fd9-9d02-7848016f7954'} Oct 19 00:51:16.290308 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 19 00:51:16.290308 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 19 00:51:16.290308 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 19 00:51:16.290308 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 19 00:51:16.290308 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 19 00:51:16.290308 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 19 00:51:16.290308 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 19 00:51:16.290308 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 19 00:51:16.290308 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 19 00:51:16.290308 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 19 00:51:16.290308 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance b38d4219-8912-4fd9-9d02-7848016f7954 could not be found. Oct 19 00:51:16.290308 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:51:16.290308 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 19 00:51:16.290308 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:51:16.290308 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 19 00:51:16.290308 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 19 00:51:16.290308 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 19 00:51:16.290308 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 19 00:51:16.290308 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 19 00:51:16.290308 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 19 00:51:16.291864 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 19 00:51:16.291864 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 19 00:51:16.291864 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 19 00:51:16.291864 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 19 00:51:16.291864 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 19 00:51:16.291864 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 19 00:51:16.291864 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 19 00:51:16.291864 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance b38d4219-8912-4fd9-9d02-7848016f7954 could not be found. Oct 19 00:51:16.291864 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:51:16.291864 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 19 00:51:16.291864 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:51:16.291864 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 19 00:51:16.291864 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 19 00:51:16.291864 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 19 00:51:16.291864 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 19 00:51:16.291864 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 19 00:51:16.291864 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance b38d4219-8912-4fd9-9d02-7848016f7954 could not be found. Oct 19 00:51:16.291864 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:51:16.294269 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-019fc280-79eb-4987-9d76-a99160d7dcbe tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] Returning 404 to user: Instance b38d4219-8912-4fd9-9d02-7848016f7954 could not be found. {{(pid=125641) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 19 00:51:16.294269 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-019fc280-79eb-4987-9d76-a99160d7dcbe tempest-AttachVolumeTestJSON-35550062 tempest-AttachVolumeTestJSON-35550062-project] 10.222.0.26 "GET /compute/v2.1/servers/b38d4219-8912-4fd9-9d02-7848016f7954" status: 404 len: 111 microversion: 2.1 time: 0.033812 Oct 19 00:51:16.294269 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 875/990] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:51:16 2022] GET /compute/v2.1/servers/b38d4219-8912-4fd9-9d02-7848016f7954 => generated 111 bytes in 35 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Oct 19 00:51:16.914112 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-940dbde4-08bb-4402-8d86-929ea7c31926 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:51:16.915445 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-940dbde4-08bb-4402-8d86-929ea7c31926 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] [instance: bfbf8641-41b8-47d8-af4d-ef17c6bd1935] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:51:16.924341 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-940dbde4-08bb-4402-8d86-929ea7c31926 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:51:16.924961 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-940dbde4-08bb-4402-8d86-929ea7c31926 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:51:16.925504 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-940dbde4-08bb-4402-8d86-929ea7c31926 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:51:16.926462 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-940dbde4-08bb-4402-8d86-929ea7c31926 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:51:16.927050 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-940dbde4-08bb-4402-8d86-929ea7c31926 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:51:16.927640 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-940dbde4-08bb-4402-8d86-929ea7c31926 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:51:17.202606 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-940dbde4-08bb-4402-8d86-929ea7c31926 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '531efd120096452caf8c0b7a7f6b9ba1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a8a6d900d9d344a694b955f51fa46aa9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:51:17.212407 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-940dbde4-08bb-4402-8d86-929ea7c31926 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] 10.222.0.26 "GET /compute/v2.1/servers/bfbf8641-41b8-47d8-af4d-ef17c6bd1935" status: 200 len: 1757 microversion: 2.1 time: 0.302501 Oct 19 00:51:17.213302 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 876/991] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:51:16 2022] GET /compute/v2.1/servers/bfbf8641-41b8-47d8-af4d-ef17c6bd1935 => generated 1757 bytes in 304 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 19 00:51:18.244190 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-a288487e-41e3-458c-9960-3c855fc8c8cf tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:51:18.245389 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-a288487e-41e3-458c-9960-3c855fc8c8cf tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] [instance: bfbf8641-41b8-47d8-af4d-ef17c6bd1935] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:51:18.259902 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-a288487e-41e3-458c-9960-3c855fc8c8cf tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:51:18.261156 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-a288487e-41e3-458c-9960-3c855fc8c8cf tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:51:18.262821 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-a288487e-41e3-458c-9960-3c855fc8c8cf tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:51:18.264155 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-a288487e-41e3-458c-9960-3c855fc8c8cf tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:51:18.265309 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-a288487e-41e3-458c-9960-3c855fc8c8cf tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:51:18.266339 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-a288487e-41e3-458c-9960-3c855fc8c8cf tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:51:18.560636 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-a288487e-41e3-458c-9960-3c855fc8c8cf tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '531efd120096452caf8c0b7a7f6b9ba1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a8a6d900d9d344a694b955f51fa46aa9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:51:18.581142 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-a288487e-41e3-458c-9960-3c855fc8c8cf tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] 10.222.0.26 "GET /compute/v2.1/servers/bfbf8641-41b8-47d8-af4d-ef17c6bd1935" status: 200 len: 1757 microversion: 2.1 time: 0.344794 Oct 19 00:51:18.583039 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 877/992] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:51:18 2022] GET /compute/v2.1/servers/bfbf8641-41b8-47d8-af4d-ef17c6bd1935 => generated 1757 bytes in 348 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:51:19.610247 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-8d5c81d5-2e67-45d4-84b2-bbf79a12bdbf tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:51:19.610247 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-8d5c81d5-2e67-45d4-84b2-bbf79a12bdbf tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] [instance: bfbf8641-41b8-47d8-af4d-ef17c6bd1935] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:51:19.619083 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-8d5c81d5-2e67-45d4-84b2-bbf79a12bdbf tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:51:19.619083 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-8d5c81d5-2e67-45d4-84b2-bbf79a12bdbf tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:51:19.619623 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-8d5c81d5-2e67-45d4-84b2-bbf79a12bdbf tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:51:19.620924 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-8d5c81d5-2e67-45d4-84b2-bbf79a12bdbf tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:51:19.621322 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-8d5c81d5-2e67-45d4-84b2-bbf79a12bdbf tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:51:19.622121 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-8d5c81d5-2e67-45d4-84b2-bbf79a12bdbf tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:51:20.000794 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-8d5c81d5-2e67-45d4-84b2-bbf79a12bdbf tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '531efd120096452caf8c0b7a7f6b9ba1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a8a6d900d9d344a694b955f51fa46aa9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:51:20.020156 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-8d5c81d5-2e67-45d4-84b2-bbf79a12bdbf tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] 10.222.0.26 "GET /compute/v2.1/servers/bfbf8641-41b8-47d8-af4d-ef17c6bd1935" status: 200 len: 1757 microversion: 2.1 time: 0.416409 Oct 19 00:51:20.021189 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 878/993] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:51:19 2022] GET /compute/v2.1/servers/bfbf8641-41b8-47d8-af4d-ef17c6bd1935 => generated 1757 bytes in 419 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 19 00:51:21.051848 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-584ab899-f05f-44c4-852e-ab15ca4cd5e9 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:51:21.053483 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-584ab899-f05f-44c4-852e-ab15ca4cd5e9 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] [instance: bfbf8641-41b8-47d8-af4d-ef17c6bd1935] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:51:21.078264 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG dbcounter [-] [125641] Writing DB stats nova_api:SELECT=70,nova_api:UPDATE=1,nova_api:DELETE=1 {{(pid=125641) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 00:51:21.078264 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-584ab899-f05f-44c4-852e-ab15ca4cd5e9 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:51:21.078264 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-584ab899-f05f-44c4-852e-ab15ca4cd5e9 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:51:21.078264 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-584ab899-f05f-44c4-852e-ab15ca4cd5e9 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:51:21.080365 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-584ab899-f05f-44c4-852e-ab15ca4cd5e9 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:51:21.080594 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-584ab899-f05f-44c4-852e-ab15ca4cd5e9 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:51:21.081418 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-584ab899-f05f-44c4-852e-ab15ca4cd5e9 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:51:21.408469 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-584ab899-f05f-44c4-852e-ab15ca4cd5e9 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '531efd120096452caf8c0b7a7f6b9ba1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a8a6d900d9d344a694b955f51fa46aa9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:51:21.425671 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-584ab899-f05f-44c4-852e-ab15ca4cd5e9 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] 10.222.0.26 "GET /compute/v2.1/servers/bfbf8641-41b8-47d8-af4d-ef17c6bd1935" status: 200 len: 1757 microversion: 2.1 time: 0.378879 Oct 19 00:51:21.426941 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 879/994] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:51:21 2022] GET /compute/v2.1/servers/bfbf8641-41b8-47d8-af4d-ef17c6bd1935 => generated 1757 bytes in 382 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:51:22.466179 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-cd3227ee-8ff2-444d-b6b8-91639f0e9562 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:51:22.467253 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-cd3227ee-8ff2-444d-b6b8-91639f0e9562 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] [instance: bfbf8641-41b8-47d8-af4d-ef17c6bd1935] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:51:22.477754 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-cd3227ee-8ff2-444d-b6b8-91639f0e9562 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:51:22.479004 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-cd3227ee-8ff2-444d-b6b8-91639f0e9562 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:51:22.479745 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-cd3227ee-8ff2-444d-b6b8-91639f0e9562 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:51:22.481028 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-cd3227ee-8ff2-444d-b6b8-91639f0e9562 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:51:22.481904 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-cd3227ee-8ff2-444d-b6b8-91639f0e9562 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:51:22.482758 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-cd3227ee-8ff2-444d-b6b8-91639f0e9562 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:51:22.702038 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-cd3227ee-8ff2-444d-b6b8-91639f0e9562 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '531efd120096452caf8c0b7a7f6b9ba1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a8a6d900d9d344a694b955f51fa46aa9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:51:22.711029 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-cd3227ee-8ff2-444d-b6b8-91639f0e9562 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] 10.222.0.26 "GET /compute/v2.1/servers/bfbf8641-41b8-47d8-af4d-ef17c6bd1935" status: 200 len: 1757 microversion: 2.1 time: 0.253154 Oct 19 00:51:22.711614 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 880/995] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:51:22 2022] GET /compute/v2.1/servers/bfbf8641-41b8-47d8-af4d-ef17c6bd1935 => generated 1757 bytes in 255 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 19 00:51:22.927418 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-c8c2c8ed-5b08-4dcd-9ed8-baa682a2df5a tempest-VolumesSnapshotsTestJSON-72094637 tempest-VolumesSnapshotsTestJSON-72094637-project] Action: 'create', calling method: .version_select at 0x7f71c38c6700>, body: {"volume": {"size": 1, "display_name": "tempest-VolumesSnapshotsTestJSON-volume-162899305"}} {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 19 00:51:23.273181 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-c8c2c8ed-5b08-4dcd-9ed8-baa682a2df5a tempest-VolumesSnapshotsTestJSON-72094637 tempest-VolumesSnapshotsTestJSON-72094637-project] 10.222.0.26 "POST /compute/v2.1/os-volumes" status: 200 len: 344 microversion: 2.1 time: 0.471369 Oct 19 00:51:23.273821 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 881/996] 10.222.0.26 () {60 vars in 1155 bytes} [Wed Oct 19 00:51:22 2022] POST /compute/v2.1/os-volumes => generated 344 bytes in 473 msecs (HTTP/1.1 200) 10 headers in 448 bytes (2 switches on core 0) Oct 19 00:51:23.290093 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-9d151421-5bd5-4bc8-9797-106cfcbde1a2 tempest-VolumesSnapshotsTestJSON-72094637 tempest-VolumesSnapshotsTestJSON-72094637-project] Calling method '.version_select at 0x7f71c38ba430>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:51:23.347494 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-9d151421-5bd5-4bc8-9797-106cfcbde1a2 tempest-VolumesSnapshotsTestJSON-72094637 tempest-VolumesSnapshotsTestJSON-72094637-project] 10.222.0.26 "GET /compute/v2.1/os-volumes/cc2416a6-24da-416f-b1dc-d3d073d178c7" status: 200 len: 344 microversion: 2.1 time: 0.061547 Oct 19 00:51:23.348268 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 882/997] 10.222.0.26 () {58 vars in 1245 bytes} [Wed Oct 19 00:51:23 2022] GET /compute/v2.1/os-volumes/cc2416a6-24da-416f-b1dc-d3d073d178c7 => generated 344 bytes in 63 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 19 00:51:23.737739 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-e3418bb6-0ff5-43b2-ada1-37cf676c2d7c tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:51:23.738927 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-e3418bb6-0ff5-43b2-ada1-37cf676c2d7c tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] [instance: bfbf8641-41b8-47d8-af4d-ef17c6bd1935] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:51:23.748441 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-e3418bb6-0ff5-43b2-ada1-37cf676c2d7c tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:51:23.748936 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-e3418bb6-0ff5-43b2-ada1-37cf676c2d7c tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:51:23.749374 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-e3418bb6-0ff5-43b2-ada1-37cf676c2d7c tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:51:23.750099 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-e3418bb6-0ff5-43b2-ada1-37cf676c2d7c tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:51:23.750593 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-e3418bb6-0ff5-43b2-ada1-37cf676c2d7c tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:51:23.751026 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-e3418bb6-0ff5-43b2-ada1-37cf676c2d7c tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:51:23.925690 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-e3418bb6-0ff5-43b2-ada1-37cf676c2d7c tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '531efd120096452caf8c0b7a7f6b9ba1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a8a6d900d9d344a694b955f51fa46aa9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:51:23.936314 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-e3418bb6-0ff5-43b2-ada1-37cf676c2d7c tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] 10.222.0.26 "GET /compute/v2.1/servers/bfbf8641-41b8-47d8-af4d-ef17c6bd1935" status: 200 len: 1757 microversion: 2.1 time: 0.204280 Oct 19 00:51:23.936917 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 883/998] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:51:23 2022] GET /compute/v2.1/servers/bfbf8641-41b8-47d8-af4d-ef17c6bd1935 => generated 1757 bytes in 207 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 19 00:51:24.363946 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-b6db3a99-8231-45de-a07e-ccb839127cad tempest-VolumesSnapshotsTestJSON-72094637 tempest-VolumesSnapshotsTestJSON-72094637-project] Calling method '.version_select at 0x7f71c388eee0>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:51:24.426905 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-b6db3a99-8231-45de-a07e-ccb839127cad tempest-VolumesSnapshotsTestJSON-72094637 tempest-VolumesSnapshotsTestJSON-72094637-project] 10.222.0.26 "GET /compute/v2.1/os-volumes/cc2416a6-24da-416f-b1dc-d3d073d178c7" status: 200 len: 345 microversion: 2.1 time: 0.066769 Oct 19 00:51:24.427600 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 884/999] 10.222.0.26 () {58 vars in 1245 bytes} [Wed Oct 19 00:51:24 2022] GET /compute/v2.1/os-volumes/cc2416a6-24da-416f-b1dc-d3d073d178c7 => generated 345 bytes in 68 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 19 00:51:24.442760 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-3278896b-b795-4d81-9486-6305c75ec3d2 tempest-VolumesSnapshotsTestJSON-72094637 tempest-VolumesSnapshotsTestJSON-72094637-project] Action: 'create', calling method: .version_select at 0x7f71c38bad30>, body: {"snapshot": {"volume_id": "cc2416a6-24da-416f-b1dc-d3d073d178c7", "display_name": "tempest-VolumesSnapshotsTestJSON-Snapshot-1084120164"}} {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 19 00:51:24.643411 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-3278896b-b795-4d81-9486-6305c75ec3d2 tempest-VolumesSnapshotsTestJSON-72094637 tempest-VolumesSnapshotsTestJSON-72094637-project] 10.222.0.26 "POST /compute/v2.1/os-snapshots" status: 200 len: 287 microversion: 2.1 time: 0.204395 Oct 19 00:51:24.644579 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 885/1000] 10.222.0.26 () {60 vars in 1162 bytes} [Wed Oct 19 00:51:24 2022] POST /compute/v2.1/os-snapshots => generated 287 bytes in 207 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 19 00:51:24.659613 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-036824da-fc9e-40ee-b0d3-8912c66171f9 tempest-VolumesSnapshotsTestJSON-72094637 tempest-VolumesSnapshotsTestJSON-72094637-project] Calling method '.version_select at 0x7f71c38c6dc0>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:51:24.698333 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-036824da-fc9e-40ee-b0d3-8912c66171f9 tempest-VolumesSnapshotsTestJSON-72094637 tempest-VolumesSnapshotsTestJSON-72094637-project] 10.222.0.26 "GET /compute/v2.1/os-snapshots/d6488746-dfde-44b8-9a4c-6ae2951445b7" status: 200 len: 287 microversion: 2.1 time: 0.044362 Oct 19 00:51:24.700615 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 886/1001] 10.222.0.26 () {58 vars in 1251 bytes} [Wed Oct 19 00:51:24 2022] GET /compute/v2.1/os-snapshots/d6488746-dfde-44b8-9a4c-6ae2951445b7 => generated 287 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 19 00:51:24.717931 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-2b03674a-4f99-44bb-af14-7b64e8c3add2 tempest-VolumesSnapshotsTestJSON-72094637 tempest-VolumesSnapshotsTestJSON-72094637-project] Calling method '.version_select at 0x7f71cb565b80>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:51:24.758087 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-2b03674a-4f99-44bb-af14-7b64e8c3add2 tempest-VolumesSnapshotsTestJSON-72094637 tempest-VolumesSnapshotsTestJSON-72094637-project] 10.222.0.26 "GET /compute/v2.1/os-snapshots" status: 200 len: 290 microversion: 2.1 time: 0.048149 Oct 19 00:51:24.759164 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 887/1002] 10.222.0.26 () {58 vars in 1140 bytes} [Wed Oct 19 00:51:24 2022] GET /compute/v2.1/os-snapshots => generated 290 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 19 00:51:24.771838 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-442a21fe-2e3d-440a-95b8-a77039fa097e tempest-VolumesSnapshotsTestJSON-72094637 tempest-VolumesSnapshotsTestJSON-72094637-project] Calling method '.version_select at 0x7f71c388ec10>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:51:24.804840 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-442a21fe-2e3d-440a-95b8-a77039fa097e tempest-VolumesSnapshotsTestJSON-72094637 tempest-VolumesSnapshotsTestJSON-72094637-project] 10.222.0.26 "GET /compute/v2.1/os-snapshots/d6488746-dfde-44b8-9a4c-6ae2951445b7" status: 200 len: 287 microversion: 2.1 time: 0.036796 Oct 19 00:51:24.805411 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 888/1003] 10.222.0.26 () {58 vars in 1251 bytes} [Wed Oct 19 00:51:24 2022] GET /compute/v2.1/os-snapshots/d6488746-dfde-44b8-9a4c-6ae2951445b7 => generated 287 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 19 00:51:24.961104 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-d606e069-b329-4cc6-88ff-2021e92cc226 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:51:24.961744 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-d606e069-b329-4cc6-88ff-2021e92cc226 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] [instance: bfbf8641-41b8-47d8-af4d-ef17c6bd1935] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:51:24.969635 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d606e069-b329-4cc6-88ff-2021e92cc226 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:51:24.970159 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d606e069-b329-4cc6-88ff-2021e92cc226 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:51:24.970640 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d606e069-b329-4cc6-88ff-2021e92cc226 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:51:24.971396 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d606e069-b329-4cc6-88ff-2021e92cc226 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:51:24.971861 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d606e069-b329-4cc6-88ff-2021e92cc226 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:51:24.972288 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d606e069-b329-4cc6-88ff-2021e92cc226 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:51:25.141234 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-d606e069-b329-4cc6-88ff-2021e92cc226 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '531efd120096452caf8c0b7a7f6b9ba1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a8a6d900d9d344a694b955f51fa46aa9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:51:25.150129 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-d606e069-b329-4cc6-88ff-2021e92cc226 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] 10.222.0.26 "GET /compute/v2.1/servers/bfbf8641-41b8-47d8-af4d-ef17c6bd1935" status: 200 len: 1757 microversion: 2.1 time: 0.193014 Oct 19 00:51:25.150765 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 889/1004] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:51:24 2022] GET /compute/v2.1/servers/bfbf8641-41b8-47d8-af4d-ef17c6bd1935 => generated 1757 bytes in 194 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 19 00:51:25.818859 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-df96caab-59e1-4a85-901a-2df3de9097db tempest-VolumesSnapshotsTestJSON-72094637 tempest-VolumesSnapshotsTestJSON-72094637-project] Calling method '.version_select at 0x7f71c388e160>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:51:25.853316 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-df96caab-59e1-4a85-901a-2df3de9097db tempest-VolumesSnapshotsTestJSON-72094637 tempest-VolumesSnapshotsTestJSON-72094637-project] 10.222.0.26 "GET /compute/v2.1/os-snapshots/d6488746-dfde-44b8-9a4c-6ae2951445b7" status: 200 len: 287 microversion: 2.1 time: 0.038432 Oct 19 00:51:25.853902 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 890/1005] 10.222.0.26 () {58 vars in 1251 bytes} [Wed Oct 19 00:51:25 2022] GET /compute/v2.1/os-snapshots/d6488746-dfde-44b8-9a4c-6ae2951445b7 => generated 287 bytes in 40 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 19 00:51:26.175764 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-c7d7cc54-9b76-4ff7-b1b2-1ed9dbd91278 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:51:26.176289 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-c7d7cc54-9b76-4ff7-b1b2-1ed9dbd91278 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] [instance: bfbf8641-41b8-47d8-af4d-ef17c6bd1935] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:51:26.182770 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-c7d7cc54-9b76-4ff7-b1b2-1ed9dbd91278 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:51:26.183829 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-c7d7cc54-9b76-4ff7-b1b2-1ed9dbd91278 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:51:26.184496 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-c7d7cc54-9b76-4ff7-b1b2-1ed9dbd91278 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:51:26.185100 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-c7d7cc54-9b76-4ff7-b1b2-1ed9dbd91278 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:51:26.185563 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-c7d7cc54-9b76-4ff7-b1b2-1ed9dbd91278 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:51:26.185993 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-c7d7cc54-9b76-4ff7-b1b2-1ed9dbd91278 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:51:26.358245 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-c7d7cc54-9b76-4ff7-b1b2-1ed9dbd91278 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '531efd120096452caf8c0b7a7f6b9ba1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a8a6d900d9d344a694b955f51fa46aa9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:51:26.367045 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-c7d7cc54-9b76-4ff7-b1b2-1ed9dbd91278 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] 10.222.0.26 "GET /compute/v2.1/servers/bfbf8641-41b8-47d8-af4d-ef17c6bd1935" status: 200 len: 1757 microversion: 2.1 time: 0.195365 Oct 19 00:51:26.367804 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 891/1006] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:51:26 2022] GET /compute/v2.1/servers/bfbf8641-41b8-47d8-af4d-ef17c6bd1935 => generated 1757 bytes in 197 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 19 00:51:26.872683 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-7a9eb921-4e55-4e49-a1de-4a37a57c3351 tempest-VolumesSnapshotsTestJSON-72094637 tempest-VolumesSnapshotsTestJSON-72094637-project] Calling method '.version_select at 0x7f71cb4fa0d0>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:51:26.907220 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-7a9eb921-4e55-4e49-a1de-4a37a57c3351 tempest-VolumesSnapshotsTestJSON-72094637 tempest-VolumesSnapshotsTestJSON-72094637-project] 10.222.0.26 "GET /compute/v2.1/os-snapshots/d6488746-dfde-44b8-9a4c-6ae2951445b7" status: 200 len: 287 microversion: 2.1 time: 0.039811 Oct 19 00:51:26.907800 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 892/1007] 10.222.0.26 () {58 vars in 1251 bytes} [Wed Oct 19 00:51:26 2022] GET /compute/v2.1/os-snapshots/d6488746-dfde-44b8-9a4c-6ae2951445b7 => generated 287 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 19 00:51:27.395171 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-ac50113a-2da3-419d-8970-84e6c6be9f40 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:51:27.396003 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-ac50113a-2da3-419d-8970-84e6c6be9f40 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] [instance: bfbf8641-41b8-47d8-af4d-ef17c6bd1935] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:51:27.402941 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ac50113a-2da3-419d-8970-84e6c6be9f40 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:51:27.403586 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ac50113a-2da3-419d-8970-84e6c6be9f40 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:51:27.404133 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ac50113a-2da3-419d-8970-84e6c6be9f40 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:51:27.404964 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ac50113a-2da3-419d-8970-84e6c6be9f40 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:51:27.405523 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ac50113a-2da3-419d-8970-84e6c6be9f40 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:51:27.406053 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ac50113a-2da3-419d-8970-84e6c6be9f40 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:51:27.591853 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-ac50113a-2da3-419d-8970-84e6c6be9f40 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '531efd120096452caf8c0b7a7f6b9ba1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a8a6d900d9d344a694b955f51fa46aa9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:51:27.607867 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-ac50113a-2da3-419d-8970-84e6c6be9f40 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] 10.222.0.26 "GET /compute/v2.1/servers/bfbf8641-41b8-47d8-af4d-ef17c6bd1935" status: 200 len: 1757 microversion: 2.1 time: 0.216935 Oct 19 00:51:27.608899 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 893/1008] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:51:27 2022] GET /compute/v2.1/servers/bfbf8641-41b8-47d8-af4d-ef17c6bd1935 => generated 1757 bytes in 219 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 19 00:51:27.921557 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-38e58b70-dc46-47d4-bc41-98bbb6ea1b4e tempest-VolumesSnapshotsTestJSON-72094637 tempest-VolumesSnapshotsTestJSON-72094637-project] Calling method '.version_select at 0x7f71cb4faf70>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:51:27.961556 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-38e58b70-dc46-47d4-bc41-98bbb6ea1b4e tempest-VolumesSnapshotsTestJSON-72094637 tempest-VolumesSnapshotsTestJSON-72094637-project] 10.222.0.26 "GET /compute/v2.1/os-snapshots/d6488746-dfde-44b8-9a4c-6ae2951445b7" status: 200 len: 288 microversion: 2.1 time: 0.045503 Oct 19 00:51:27.962265 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 894/1009] 10.222.0.26 () {58 vars in 1251 bytes} [Wed Oct 19 00:51:27 2022] GET /compute/v2.1/os-snapshots/d6488746-dfde-44b8-9a4c-6ae2951445b7 => generated 288 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 19 00:51:27.977238 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-9e8ec948-74aa-42f3-a385-bb58b75fccca tempest-VolumesSnapshotsTestJSON-72094637 tempest-VolumesSnapshotsTestJSON-72094637-project] Calling method '.version_select at 0x7f71c38bab80>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:51:28.082669 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-9e8ec948-74aa-42f3-a385-bb58b75fccca tempest-VolumesSnapshotsTestJSON-72094637 tempest-VolumesSnapshotsTestJSON-72094637-project] 10.222.0.26 "DELETE /compute/v2.1/os-snapshots/d6488746-dfde-44b8-9a4c-6ae2951445b7" status: 202 len: 0 microversion: 2.1 time: 0.111028 Oct 19 00:51:28.083672 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 895/1010] 10.222.0.26 () {58 vars in 1254 bytes} [Wed Oct 19 00:51:27 2022] DELETE /compute/v2.1/os-snapshots/d6488746-dfde-44b8-9a4c-6ae2951445b7 => generated 0 bytes in 113 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Oct 19 00:51:28.094692 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-576c75ec-1b96-4936-ba86-125d62ce336a tempest-VolumesSnapshotsTestJSON-72094637 tempest-VolumesSnapshotsTestJSON-72094637-project] Calling method '.version_select at 0x7f71c39264c0>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:51:28.132970 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-576c75ec-1b96-4936-ba86-125d62ce336a tempest-VolumesSnapshotsTestJSON-72094637 tempest-VolumesSnapshotsTestJSON-72094637-project] 10.222.0.26 "GET /compute/v2.1/os-snapshots/d6488746-dfde-44b8-9a4c-6ae2951445b7" status: 200 len: 287 microversion: 2.1 time: 0.044392 Oct 19 00:51:28.133657 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 896/1011] 10.222.0.26 () {58 vars in 1251 bytes} [Wed Oct 19 00:51:28 2022] GET /compute/v2.1/os-snapshots/d6488746-dfde-44b8-9a4c-6ae2951445b7 => generated 287 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 19 00:51:28.645765 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-d8f2e0f8-50f5-4d0d-ad29-8d2112318ef3 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:51:28.646397 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-d8f2e0f8-50f5-4d0d-ad29-8d2112318ef3 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] [instance: bfbf8641-41b8-47d8-af4d-ef17c6bd1935] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:51:28.654070 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d8f2e0f8-50f5-4d0d-ad29-8d2112318ef3 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:51:28.654592 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d8f2e0f8-50f5-4d0d-ad29-8d2112318ef3 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:51:28.655037 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d8f2e0f8-50f5-4d0d-ad29-8d2112318ef3 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:51:28.655823 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d8f2e0f8-50f5-4d0d-ad29-8d2112318ef3 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:51:28.656282 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d8f2e0f8-50f5-4d0d-ad29-8d2112318ef3 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:51:28.656708 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d8f2e0f8-50f5-4d0d-ad29-8d2112318ef3 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:51:28.828658 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-d8f2e0f8-50f5-4d0d-ad29-8d2112318ef3 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '531efd120096452caf8c0b7a7f6b9ba1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a8a6d900d9d344a694b955f51fa46aa9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:51:28.837679 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-d8f2e0f8-50f5-4d0d-ad29-8d2112318ef3 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] 10.222.0.26 "GET /compute/v2.1/servers/bfbf8641-41b8-47d8-af4d-ef17c6bd1935" status: 200 len: 1760 microversion: 2.1 time: 0.195799 Oct 19 00:51:28.838275 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 897/1012] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:51:28 2022] GET /compute/v2.1/servers/bfbf8641-41b8-47d8-af4d-ef17c6bd1935 => generated 1760 bytes in 197 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 19 00:51:29.149881 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-12c28826-3059-4b91-bab8-c5e3b3bb8dbe tempest-VolumesSnapshotsTestJSON-72094637 tempest-VolumesSnapshotsTestJSON-72094637-project] Calling method '.version_select at 0x7f71c39268b0>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:51:29.179882 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.wsgi [None req-12c28826-3059-4b91-bab8-c5e3b3bb8dbe tempest-VolumesSnapshotsTestJSON-72094637 tempest-VolumesSnapshotsTestJSON-72094637-project] HTTP exception thrown: Snapshot d6488746-dfde-44b8-9a4c-6ae2951445b7 could not be found. Oct 19 00:51:29.182213 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-12c28826-3059-4b91-bab8-c5e3b3bb8dbe tempest-VolumesSnapshotsTestJSON-72094637 tempest-VolumesSnapshotsTestJSON-72094637-project] Request method failure captured: Oct 19 00:51:29.182213 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: request: GET /compute/v2.1/os-snapshots/d6488746-dfde-44b8-9a4c-6ae2951445b7 HTTP/1.1 Oct 19 00:51:29.182213 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Accept: application/json Oct 19 00:51:29.182213 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Accept-Encoding: identity Oct 19 00:51:29.182213 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Connection: close Oct 19 00:51:29.182213 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Content-Length: 0 Oct 19 00:51:29.182213 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Content-Type: application/json Oct 19 00:51:29.182213 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Host: 10.222.0.26 Oct 19 00:51:29.182213 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Openstack-System-Scope: None Oct 19 00:51:29.182213 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: User-Agent: python-urllib3/1.26.12 Oct 19 00:51:29.182213 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Auth-Token: gAAAAABjT0oKeOJNR8q9aWkGv3p6MAYH7RF6lgA37svGLOBNuAIJ7PLpGDrlGkj23KyZ0lzGg8d9KmuLgEjdgJPe-Y1tHlbu7H-xTwJPt07WPtNQmrISUVixny3w77mRpBzbRE1mq_tIBYL4_q1F198XpLVtAvgUOTeieYNJMMBUrUqipLdT8DA Oct 19 00:51:29.182213 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Domain-Id: None Oct 19 00:51:29.182213 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Domain-Name: None Oct 19 00:51:29.182213 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Identity-Status: Confirmed Oct 19 00:51:29.182213 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Is-Admin-Project: True Oct 19 00:51:29.182213 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Project-Domain-Id: default Oct 19 00:51:29.182213 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Project-Domain-Name: Default Oct 19 00:51:29.182213 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Project-Id: 7e8918ddd84e4f1a9c3e0003ae181c1a Oct 19 00:51:29.182213 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Project-Name: tempest-VolumesSnapshotsTestJSON-72094637 Oct 19 00:51:29.182213 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Role: member,reader Oct 19 00:51:29.182213 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Roles: member,reader Oct 19 00:51:29.185303 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/7e8918ddd84e4f1a9c3e0003ae181c1a"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/7e8918ddd84e4f1a9c3e0003ae181c1a"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/7e8918ddd84e4f1a9c3e0003ae181c1a"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_7e8918ddd84e4f1a9c3e0003ae181c1a"}]}] Oct 19 00:51:29.185303 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Tenant: tempest-VolumesSnapshotsTestJSON-72094637 Oct 19 00:51:29.185303 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Tenant-Id: 7e8918ddd84e4f1a9c3e0003ae181c1a Oct 19 00:51:29.185303 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Tenant-Name: tempest-VolumesSnapshotsTestJSON-72094637 Oct 19 00:51:29.185303 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User: tempest-VolumesSnapshotsTestJSON-72094637-project Oct 19 00:51:29.185303 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User-Domain-Id: default Oct 19 00:51:29.185303 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User-Domain-Name: Default Oct 19 00:51:29.185303 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User-Id: dc1a31109df44058a2f739f310ddf87b Oct 19 00:51:29.185303 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User-Name: tempest-VolumesSnapshotsTestJSON-72094637-project Oct 19 00:51:29.185303 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: method: Oct 19 00:51:29.185303 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: exception: Snapshot d6488746-dfde-44b8-9a4c-6ae2951445b7 could not be found. Oct 19 00:51:29.185303 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: action_args: {'id': 'd6488746-dfde-44b8-9a4c-6ae2951445b7'} Oct 19 00:51:29.185303 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 19 00:51:29.187012 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 19 00:51:29.187012 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 464, in wrapper Oct 19 00:51:29.187012 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi res = method(self, ctx, snapshot_id, *args, **kwargs) Oct 19 00:51:29.187012 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 701, in get_snapshot Oct 19 00:51:29.187012 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi item = cinderclient(context).volume_snapshots.get(snapshot_id) Oct 19 00:51:29.187012 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volume_snapshots.py", line 160, in get Oct 19 00:51:29.187012 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi return self._get("/snapshots/%s" % snapshot_id, "snapshot") Oct 19 00:51:29.187012 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Oct 19 00:51:29.187012 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Oct 19 00:51:29.187012 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Oct 19 00:51:29.187012 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Oct 19 00:51:29.187012 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Oct 19 00:51:29.187012 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Oct 19 00:51:29.187012 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Oct 19 00:51:29.187012 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Oct 19 00:51:29.187012 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Snapshot d6488746-dfde-44b8-9a4c-6ae2951445b7 could not be found. (HTTP 404) (Request-ID: req-41bccffd-075e-4247-95a1-c7cc4d335b63) Oct 19 00:51:29.187012 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:51:29.187012 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 19 00:51:29.189488 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:51:29.189488 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 19 00:51:29.189488 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 617, in show Oct 19 00:51:29.189488 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi vol = self.volume_api.get_snapshot(context, id) Oct 19 00:51:29.189488 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Oct 19 00:51:29.189488 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Oct 19 00:51:29.189488 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 466, in wrapper Oct 19 00:51:29.189488 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi _reraise(exception.SnapshotNotFound(snapshot_id=snapshot_id)) Oct 19 00:51:29.189488 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Oct 19 00:51:29.189488 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Oct 19 00:51:29.189488 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 464, in wrapper Oct 19 00:51:29.189488 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi res = method(self, ctx, snapshot_id, *args, **kwargs) Oct 19 00:51:29.189488 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 701, in get_snapshot Oct 19 00:51:29.189488 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi item = cinderclient(context).volume_snapshots.get(snapshot_id) Oct 19 00:51:29.189488 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volume_snapshots.py", line 160, in get Oct 19 00:51:29.189488 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi return self._get("/snapshots/%s" % snapshot_id, "snapshot") Oct 19 00:51:29.189488 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Oct 19 00:51:29.189488 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Oct 19 00:51:29.189488 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Oct 19 00:51:29.191689 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Oct 19 00:51:29.191689 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Oct 19 00:51:29.191689 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Oct 19 00:51:29.191689 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Oct 19 00:51:29.191689 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Oct 19 00:51:29.191689 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi nova.exception.SnapshotNotFound: Snapshot d6488746-dfde-44b8-9a4c-6ae2951445b7 could not be found. Oct 19 00:51:29.191689 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:51:29.191689 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 19 00:51:29.191689 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:51:29.191689 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 19 00:51:29.191689 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 19 00:51:29.191689 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 19 00:51:29.191689 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 19 00:51:29.191689 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 19 00:51:29.191689 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 19 00:51:29.191689 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 19 00:51:29.191689 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 19 00:51:29.191689 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 19 00:51:29.191689 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 619, in show Oct 19 00:51:29.191689 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 19 00:51:29.193489 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Snapshot d6488746-dfde-44b8-9a4c-6ae2951445b7 could not be found. Oct 19 00:51:29.193489 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:51:29.193489 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 19 00:51:29.193489 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:51:29.193489 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 19 00:51:29.193489 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 19 00:51:29.193489 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 19 00:51:29.193489 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 19 00:51:29.193489 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 19 00:51:29.193489 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Snapshot d6488746-dfde-44b8-9a4c-6ae2951445b7 could not be found. Oct 19 00:51:29.193489 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:51:29.193489 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-12c28826-3059-4b91-bab8-c5e3b3bb8dbe tempest-VolumesSnapshotsTestJSON-72094637 tempest-VolumesSnapshotsTestJSON-72094637-project] Returning 404 to user: Snapshot d6488746-dfde-44b8-9a4c-6ae2951445b7 could not be found. {{(pid=125641) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 19 00:51:29.193489 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-12c28826-3059-4b91-bab8-c5e3b3bb8dbe tempest-VolumesSnapshotsTestJSON-72094637 tempest-VolumesSnapshotsTestJSON-72094637-project] 10.222.0.26 "GET /compute/v2.1/os-snapshots/d6488746-dfde-44b8-9a4c-6ae2951445b7" status: 404 len: 111 microversion: 2.1 time: 0.038090 Oct 19 00:51:29.193489 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 898/1013] 10.222.0.26 () {58 vars in 1251 bytes} [Wed Oct 19 00:51:29 2022] GET /compute/v2.1/os-snapshots/d6488746-dfde-44b8-9a4c-6ae2951445b7 => generated 111 bytes in 39 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Oct 19 00:51:29.195325 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-f4bda2ca-3702-45e7-a750-b74f8e877b7c tempest-VolumesSnapshotsTestJSON-72094637 tempest-VolumesSnapshotsTestJSON-72094637-project] Calling method '.version_select at 0x7f71cb4ff040>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:51:29.264624 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-f4bda2ca-3702-45e7-a750-b74f8e877b7c tempest-VolumesSnapshotsTestJSON-72094637 tempest-VolumesSnapshotsTestJSON-72094637-project] 10.222.0.26 "DELETE /compute/v2.1/os-volumes/cc2416a6-24da-416f-b1dc-d3d073d178c7" status: 202 len: 0 microversion: 2.1 time: 0.075559 Oct 19 00:51:29.265315 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 899/1014] 10.222.0.26 () {58 vars in 1248 bytes} [Wed Oct 19 00:51:29 2022] DELETE /compute/v2.1/os-volumes/cc2416a6-24da-416f-b1dc-d3d073d178c7 => generated 0 bytes in 77 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Oct 19 00:51:29.275318 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-d2cbcf8c-2934-442c-a43b-cea9932d6b56 tempest-VolumesSnapshotsTestJSON-72094637 tempest-VolumesSnapshotsTestJSON-72094637-project] Calling method '.version_select at 0x7f71cb565b80>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:51:29.341109 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-d2cbcf8c-2934-442c-a43b-cea9932d6b56 tempest-VolumesSnapshotsTestJSON-72094637 tempest-VolumesSnapshotsTestJSON-72094637-project] 10.222.0.26 "GET /compute/v2.1/os-volumes/cc2416a6-24da-416f-b1dc-d3d073d178c7" status: 200 len: 344 microversion: 2.1 time: 0.070613 Oct 19 00:51:29.341827 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 900/1015] 10.222.0.26 () {58 vars in 1245 bytes} [Wed Oct 19 00:51:29 2022] GET /compute/v2.1/os-volumes/cc2416a6-24da-416f-b1dc-d3d073d178c7 => generated 344 bytes in 72 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 19 00:51:30.357387 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-d3ce2415-e392-40c4-b034-c4c3a28b8d4b tempest-VolumesSnapshotsTestJSON-72094637 tempest-VolumesSnapshotsTestJSON-72094637-project] Calling method '.version_select at 0x7f71cb4ffd30>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:51:30.394007 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.wsgi [None req-d3ce2415-e392-40c4-b034-c4c3a28b8d4b tempest-VolumesSnapshotsTestJSON-72094637 tempest-VolumesSnapshotsTestJSON-72094637-project] HTTP exception thrown: Volume cc2416a6-24da-416f-b1dc-d3d073d178c7 could not be found. Oct 19 00:51:30.396114 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-d3ce2415-e392-40c4-b034-c4c3a28b8d4b tempest-VolumesSnapshotsTestJSON-72094637 tempest-VolumesSnapshotsTestJSON-72094637-project] Request method failure captured: Oct 19 00:51:30.396114 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: request: GET /compute/v2.1/os-volumes/cc2416a6-24da-416f-b1dc-d3d073d178c7 HTTP/1.1 Oct 19 00:51:30.396114 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Accept: application/json Oct 19 00:51:30.396114 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Accept-Encoding: identity Oct 19 00:51:30.396114 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Connection: close Oct 19 00:51:30.396114 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Content-Length: 0 Oct 19 00:51:30.396114 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Content-Type: application/json Oct 19 00:51:30.396114 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Host: 10.222.0.26 Oct 19 00:51:30.396114 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Openstack-System-Scope: None Oct 19 00:51:30.396114 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: User-Agent: python-urllib3/1.26.12 Oct 19 00:51:30.396114 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Auth-Token: gAAAAABjT0oKeOJNR8q9aWkGv3p6MAYH7RF6lgA37svGLOBNuAIJ7PLpGDrlGkj23KyZ0lzGg8d9KmuLgEjdgJPe-Y1tHlbu7H-xTwJPt07WPtNQmrISUVixny3w77mRpBzbRE1mq_tIBYL4_q1F198XpLVtAvgUOTeieYNJMMBUrUqipLdT8DA Oct 19 00:51:30.396114 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Domain-Id: None Oct 19 00:51:30.396114 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Domain-Name: None Oct 19 00:51:30.396114 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Identity-Status: Confirmed Oct 19 00:51:30.396114 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Is-Admin-Project: True Oct 19 00:51:30.396114 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Project-Domain-Id: default Oct 19 00:51:30.396114 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Project-Domain-Name: Default Oct 19 00:51:30.396114 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Project-Id: 7e8918ddd84e4f1a9c3e0003ae181c1a Oct 19 00:51:30.396114 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Project-Name: tempest-VolumesSnapshotsTestJSON-72094637 Oct 19 00:51:30.396114 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Role: member,reader Oct 19 00:51:30.396114 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Roles: member,reader Oct 19 00:51:30.398643 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/7e8918ddd84e4f1a9c3e0003ae181c1a"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/7e8918ddd84e4f1a9c3e0003ae181c1a"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/7e8918ddd84e4f1a9c3e0003ae181c1a"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_7e8918ddd84e4f1a9c3e0003ae181c1a"}]}] Oct 19 00:51:30.398643 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Tenant: tempest-VolumesSnapshotsTestJSON-72094637 Oct 19 00:51:30.398643 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Tenant-Id: 7e8918ddd84e4f1a9c3e0003ae181c1a Oct 19 00:51:30.398643 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Tenant-Name: tempest-VolumesSnapshotsTestJSON-72094637 Oct 19 00:51:30.398643 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User: tempest-VolumesSnapshotsTestJSON-72094637-project Oct 19 00:51:30.398643 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User-Domain-Id: default Oct 19 00:51:30.398643 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User-Domain-Name: Default Oct 19 00:51:30.398643 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User-Id: dc1a31109df44058a2f739f310ddf87b Oct 19 00:51:30.398643 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User-Name: tempest-VolumesSnapshotsTestJSON-72094637-project Oct 19 00:51:30.398643 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: method: Oct 19 00:51:30.398643 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: exception: Volume cc2416a6-24da-416f-b1dc-d3d073d178c7 could not be found. Oct 19 00:51:30.398643 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: action_args: {'id': 'cc2416a6-24da-416f-b1dc-d3d073d178c7'} Oct 19 00:51:30.398643 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 19 00:51:30.400153 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 19 00:51:30.400153 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Oct 19 00:51:30.400153 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Oct 19 00:51:30.400153 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Oct 19 00:51:30.400153 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi item = cinderclient( Oct 19 00:51:30.400153 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Oct 19 00:51:30.400153 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Oct 19 00:51:30.400153 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Oct 19 00:51:30.400153 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Oct 19 00:51:30.400153 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Oct 19 00:51:30.400153 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Oct 19 00:51:30.400153 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Oct 19 00:51:30.400153 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Oct 19 00:51:30.400153 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Oct 19 00:51:30.400153 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Oct 19 00:51:30.400153 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume cc2416a6-24da-416f-b1dc-d3d073d178c7 could not be found. (HTTP 404) (Request-ID: req-c42f4e43-8fce-42bf-8b65-1e3eca91ed57) Oct 19 00:51:30.400153 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:51:30.400153 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 19 00:51:30.400153 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:51:30.402363 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 19 00:51:30.402363 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 120, in show Oct 19 00:51:30.402363 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi vol = self.volume_api.get(context, id) Oct 19 00:51:30.402363 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Oct 19 00:51:30.402363 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Oct 19 00:51:30.402363 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Oct 19 00:51:30.402363 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Oct 19 00:51:30.402363 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Oct 19 00:51:30.402363 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Oct 19 00:51:30.402363 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Oct 19 00:51:30.402363 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Oct 19 00:51:30.402363 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Oct 19 00:51:30.402363 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi item = cinderclient( Oct 19 00:51:30.402363 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Oct 19 00:51:30.402363 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Oct 19 00:51:30.402363 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Oct 19 00:51:30.402363 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Oct 19 00:51:30.402363 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Oct 19 00:51:30.402363 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Oct 19 00:51:30.404732 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Oct 19 00:51:30.404732 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Oct 19 00:51:30.404732 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Oct 19 00:51:30.404732 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Oct 19 00:51:30.404732 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume cc2416a6-24da-416f-b1dc-d3d073d178c7 could not be found. Oct 19 00:51:30.404732 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:51:30.404732 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 19 00:51:30.404732 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:51:30.404732 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 19 00:51:30.404732 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 19 00:51:30.404732 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 19 00:51:30.404732 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 19 00:51:30.404732 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 19 00:51:30.404732 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 19 00:51:30.404732 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 19 00:51:30.404732 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 19 00:51:30.404732 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 19 00:51:30.404732 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 122, in show Oct 19 00:51:30.404732 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 19 00:51:30.407020 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume cc2416a6-24da-416f-b1dc-d3d073d178c7 could not be found. Oct 19 00:51:30.407020 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:51:30.407020 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 19 00:51:30.407020 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:51:30.407020 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 19 00:51:30.407020 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 19 00:51:30.407020 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 19 00:51:30.407020 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 19 00:51:30.407020 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 19 00:51:30.407020 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume cc2416a6-24da-416f-b1dc-d3d073d178c7 could not be found. Oct 19 00:51:30.407020 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:51:30.407020 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-d3ce2415-e392-40c4-b034-c4c3a28b8d4b tempest-VolumesSnapshotsTestJSON-72094637 tempest-VolumesSnapshotsTestJSON-72094637-project] Returning 404 to user: Volume cc2416a6-24da-416f-b1dc-d3d073d178c7 could not be found. {{(pid=125641) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 19 00:51:30.407020 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-d3ce2415-e392-40c4-b034-c4c3a28b8d4b tempest-VolumesSnapshotsTestJSON-72094637 tempest-VolumesSnapshotsTestJSON-72094637-project] 10.222.0.26 "GET /compute/v2.1/os-volumes/cc2416a6-24da-416f-b1dc-d3d073d178c7" status: 404 len: 109 microversion: 2.1 time: 0.044759 Oct 19 00:51:30.407020 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 901/1016] 10.222.0.26 () {58 vars in 1245 bytes} [Wed Oct 19 00:51:30 2022] GET /compute/v2.1/os-volumes/cc2416a6-24da-416f-b1dc-d3d073d178c7 => generated 109 bytes in 46 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Oct 19 00:51:30.431824 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-64e44766-42d4-458e-88f5-c58605bb708f tempest-VolumesSnapshotsTestJSON-72094637 tempest-VolumesSnapshotsTestJSON-72094637-project] Calling method '.version_select at 0x7f71c3ff11f0>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:51:30.471574 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.wsgi [None req-64e44766-42d4-458e-88f5-c58605bb708f tempest-VolumesSnapshotsTestJSON-72094637 tempest-VolumesSnapshotsTestJSON-72094637-project] HTTP exception thrown: Volume cc2416a6-24da-416f-b1dc-d3d073d178c7 could not be found. Oct 19 00:51:30.473621 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-64e44766-42d4-458e-88f5-c58605bb708f tempest-VolumesSnapshotsTestJSON-72094637 tempest-VolumesSnapshotsTestJSON-72094637-project] Request method failure captured: Oct 19 00:51:30.473621 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: request: DELETE /compute/v2.1/os-volumes/cc2416a6-24da-416f-b1dc-d3d073d178c7 HTTP/1.1 Oct 19 00:51:30.473621 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Accept: application/json Oct 19 00:51:30.473621 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Accept-Encoding: identity Oct 19 00:51:30.473621 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Connection: close Oct 19 00:51:30.473621 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Content-Length: 0 Oct 19 00:51:30.473621 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Content-Type: application/json Oct 19 00:51:30.473621 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Host: 10.222.0.26 Oct 19 00:51:30.473621 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Openstack-System-Scope: None Oct 19 00:51:30.473621 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: User-Agent: python-urllib3/1.26.12 Oct 19 00:51:30.473621 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Auth-Token: gAAAAABjT0oKeOJNR8q9aWkGv3p6MAYH7RF6lgA37svGLOBNuAIJ7PLpGDrlGkj23KyZ0lzGg8d9KmuLgEjdgJPe-Y1tHlbu7H-xTwJPt07WPtNQmrISUVixny3w77mRpBzbRE1mq_tIBYL4_q1F198XpLVtAvgUOTeieYNJMMBUrUqipLdT8DA Oct 19 00:51:30.473621 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Domain-Id: None Oct 19 00:51:30.473621 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Domain-Name: None Oct 19 00:51:30.473621 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Identity-Status: Confirmed Oct 19 00:51:30.473621 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Is-Admin-Project: True Oct 19 00:51:30.473621 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Project-Domain-Id: default Oct 19 00:51:30.473621 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Project-Domain-Name: Default Oct 19 00:51:30.473621 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Project-Id: 7e8918ddd84e4f1a9c3e0003ae181c1a Oct 19 00:51:30.473621 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Project-Name: tempest-VolumesSnapshotsTestJSON-72094637 Oct 19 00:51:30.473621 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Role: member,reader Oct 19 00:51:30.473621 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Roles: member,reader Oct 19 00:51:30.476182 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/7e8918ddd84e4f1a9c3e0003ae181c1a"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/7e8918ddd84e4f1a9c3e0003ae181c1a"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/7e8918ddd84e4f1a9c3e0003ae181c1a"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_7e8918ddd84e4f1a9c3e0003ae181c1a"}]}] Oct 19 00:51:30.476182 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Tenant: tempest-VolumesSnapshotsTestJSON-72094637 Oct 19 00:51:30.476182 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Tenant-Id: 7e8918ddd84e4f1a9c3e0003ae181c1a Oct 19 00:51:30.476182 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Tenant-Name: tempest-VolumesSnapshotsTestJSON-72094637 Oct 19 00:51:30.476182 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User: tempest-VolumesSnapshotsTestJSON-72094637-project Oct 19 00:51:30.476182 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User-Domain-Id: default Oct 19 00:51:30.476182 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User-Domain-Name: Default Oct 19 00:51:30.476182 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User-Id: dc1a31109df44058a2f739f310ddf87b Oct 19 00:51:30.476182 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User-Name: tempest-VolumesSnapshotsTestJSON-72094637-project Oct 19 00:51:30.476182 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: method: Oct 19 00:51:30.476182 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: exception: Volume cc2416a6-24da-416f-b1dc-d3d073d178c7 could not be found. Oct 19 00:51:30.476182 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: action_args: {'id': 'cc2416a6-24da-416f-b1dc-d3d073d178c7'} Oct 19 00:51:30.476182 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 19 00:51:30.477835 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 19 00:51:30.477835 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Oct 19 00:51:30.477835 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Oct 19 00:51:30.477835 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete Oct 19 00:51:30.477835 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) Oct 19 00:51:30.477835 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete Oct 19 00:51:30.477835 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi return self._delete(loc) Oct 19 00:51:30.477835 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete Oct 19 00:51:30.477835 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) Oct 19 00:51:30.477835 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete Oct 19 00:51:30.477835 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) Oct 19 00:51:30.477835 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Oct 19 00:51:30.477835 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Oct 19 00:51:30.477835 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Oct 19 00:51:30.477835 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Oct 19 00:51:30.477835 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume cc2416a6-24da-416f-b1dc-d3d073d178c7 could not be found. (HTTP 404) (Request-ID: req-2caaf79d-ec56-4737-a497-ac1f30d16a79) Oct 19 00:51:30.477835 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:51:30.477835 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 19 00:51:30.477835 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:51:30.480620 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 19 00:51:30.480620 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 136, in delete Oct 19 00:51:30.480620 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi self.volume_api.delete(context, id) Oct 19 00:51:30.480620 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Oct 19 00:51:30.480620 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Oct 19 00:51:30.480620 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Oct 19 00:51:30.480620 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Oct 19 00:51:30.480620 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Oct 19 00:51:30.480620 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Oct 19 00:51:30.480620 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Oct 19 00:51:30.480620 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Oct 19 00:51:30.480620 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete Oct 19 00:51:30.480620 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) Oct 19 00:51:30.480620 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete Oct 19 00:51:30.480620 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi return self._delete(loc) Oct 19 00:51:30.480620 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete Oct 19 00:51:30.480620 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) Oct 19 00:51:30.480620 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete Oct 19 00:51:30.480620 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) Oct 19 00:51:30.482937 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Oct 19 00:51:30.482937 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Oct 19 00:51:30.482937 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Oct 19 00:51:30.482937 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Oct 19 00:51:30.482937 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume cc2416a6-24da-416f-b1dc-d3d073d178c7 could not be found. Oct 19 00:51:30.482937 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:51:30.482937 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 19 00:51:30.482937 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:51:30.482937 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 19 00:51:30.482937 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 19 00:51:30.482937 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 19 00:51:30.482937 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 19 00:51:30.482937 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 19 00:51:30.482937 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 19 00:51:30.482937 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 19 00:51:30.482937 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 19 00:51:30.482937 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 19 00:51:30.482937 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 140, in delete Oct 19 00:51:30.482937 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 19 00:51:30.485468 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume cc2416a6-24da-416f-b1dc-d3d073d178c7 could not be found. Oct 19 00:51:30.485468 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:51:30.485468 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 19 00:51:30.485468 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:51:30.485468 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 19 00:51:30.485468 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 19 00:51:30.485468 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 19 00:51:30.485468 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 19 00:51:30.485468 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 19 00:51:30.485468 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume cc2416a6-24da-416f-b1dc-d3d073d178c7 could not be found. Oct 19 00:51:30.485468 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:51:30.485468 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-64e44766-42d4-458e-88f5-c58605bb708f tempest-VolumesSnapshotsTestJSON-72094637 tempest-VolumesSnapshotsTestJSON-72094637-project] Returning 404 to user: Volume cc2416a6-24da-416f-b1dc-d3d073d178c7 could not be found. {{(pid=125641) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 19 00:51:30.485468 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-64e44766-42d4-458e-88f5-c58605bb708f tempest-VolumesSnapshotsTestJSON-72094637 tempest-VolumesSnapshotsTestJSON-72094637-project] 10.222.0.26 "DELETE /compute/v2.1/os-volumes/cc2416a6-24da-416f-b1dc-d3d073d178c7" status: 404 len: 109 microversion: 2.1 time: 0.056400 Oct 19 00:51:30.485468 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 902/1017] 10.222.0.26 () {58 vars in 1248 bytes} [Wed Oct 19 00:51:30 2022] DELETE /compute/v2.1/os-volumes/cc2416a6-24da-416f-b1dc-d3d073d178c7 => generated 109 bytes in 59 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 19 00:51:30.489020 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-45731673-bc7c-4b9c-9f00-d991ede5a19f tempest-VolumesSnapshotsTestJSON-72094637 tempest-VolumesSnapshotsTestJSON-72094637-project] Calling method '.version_select at 0x7f71cb4ffca0>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:51:30.519199 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.wsgi [None req-45731673-bc7c-4b9c-9f00-d991ede5a19f tempest-VolumesSnapshotsTestJSON-72094637 tempest-VolumesSnapshotsTestJSON-72094637-project] HTTP exception thrown: Volume cc2416a6-24da-416f-b1dc-d3d073d178c7 could not be found. Oct 19 00:51:30.521419 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-45731673-bc7c-4b9c-9f00-d991ede5a19f tempest-VolumesSnapshotsTestJSON-72094637 tempest-VolumesSnapshotsTestJSON-72094637-project] Request method failure captured: Oct 19 00:51:30.521419 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: request: GET /compute/v2.1/os-volumes/cc2416a6-24da-416f-b1dc-d3d073d178c7 HTTP/1.1 Oct 19 00:51:30.521419 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Accept: application/json Oct 19 00:51:30.521419 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Accept-Encoding: identity Oct 19 00:51:30.521419 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Connection: close Oct 19 00:51:30.521419 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Content-Length: 0 Oct 19 00:51:30.521419 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Content-Type: application/json Oct 19 00:51:30.521419 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Host: 10.222.0.26 Oct 19 00:51:30.521419 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Openstack-System-Scope: None Oct 19 00:51:30.521419 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: User-Agent: python-urllib3/1.26.12 Oct 19 00:51:30.521419 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Auth-Token: gAAAAABjT0oKeOJNR8q9aWkGv3p6MAYH7RF6lgA37svGLOBNuAIJ7PLpGDrlGkj23KyZ0lzGg8d9KmuLgEjdgJPe-Y1tHlbu7H-xTwJPt07WPtNQmrISUVixny3w77mRpBzbRE1mq_tIBYL4_q1F198XpLVtAvgUOTeieYNJMMBUrUqipLdT8DA Oct 19 00:51:30.521419 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Domain-Id: None Oct 19 00:51:30.521419 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Domain-Name: None Oct 19 00:51:30.521419 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Identity-Status: Confirmed Oct 19 00:51:30.521419 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Is-Admin-Project: True Oct 19 00:51:30.521419 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Project-Domain-Id: default Oct 19 00:51:30.521419 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Project-Domain-Name: Default Oct 19 00:51:30.521419 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Project-Id: 7e8918ddd84e4f1a9c3e0003ae181c1a Oct 19 00:51:30.521419 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Project-Name: tempest-VolumesSnapshotsTestJSON-72094637 Oct 19 00:51:30.521419 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Role: member,reader Oct 19 00:51:30.521419 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Roles: member,reader Oct 19 00:51:30.524595 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/7e8918ddd84e4f1a9c3e0003ae181c1a"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/7e8918ddd84e4f1a9c3e0003ae181c1a"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/7e8918ddd84e4f1a9c3e0003ae181c1a"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_7e8918ddd84e4f1a9c3e0003ae181c1a"}]}] Oct 19 00:51:30.524595 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Tenant: tempest-VolumesSnapshotsTestJSON-72094637 Oct 19 00:51:30.524595 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Tenant-Id: 7e8918ddd84e4f1a9c3e0003ae181c1a Oct 19 00:51:30.524595 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Tenant-Name: tempest-VolumesSnapshotsTestJSON-72094637 Oct 19 00:51:30.524595 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User: tempest-VolumesSnapshotsTestJSON-72094637-project Oct 19 00:51:30.524595 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User-Domain-Id: default Oct 19 00:51:30.524595 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User-Domain-Name: Default Oct 19 00:51:30.524595 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User-Id: dc1a31109df44058a2f739f310ddf87b Oct 19 00:51:30.524595 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User-Name: tempest-VolumesSnapshotsTestJSON-72094637-project Oct 19 00:51:30.524595 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: method: Oct 19 00:51:30.524595 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: exception: Volume cc2416a6-24da-416f-b1dc-d3d073d178c7 could not be found. Oct 19 00:51:30.524595 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: action_args: {'id': 'cc2416a6-24da-416f-b1dc-d3d073d178c7'} Oct 19 00:51:30.524595 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 19 00:51:30.526549 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 19 00:51:30.526549 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Oct 19 00:51:30.526549 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Oct 19 00:51:30.526549 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Oct 19 00:51:30.526549 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi item = cinderclient( Oct 19 00:51:30.526549 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Oct 19 00:51:30.526549 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Oct 19 00:51:30.526549 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Oct 19 00:51:30.526549 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Oct 19 00:51:30.526549 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Oct 19 00:51:30.526549 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Oct 19 00:51:30.526549 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Oct 19 00:51:30.526549 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Oct 19 00:51:30.526549 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Oct 19 00:51:30.526549 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Oct 19 00:51:30.526549 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume cc2416a6-24da-416f-b1dc-d3d073d178c7 could not be found. (HTTP 404) (Request-ID: req-9aa6d892-08fc-4cb4-9adc-6acca0e807dd) Oct 19 00:51:30.526549 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:51:30.526549 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 19 00:51:30.526549 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:51:30.529137 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 19 00:51:30.529137 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 120, in show Oct 19 00:51:30.529137 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi vol = self.volume_api.get(context, id) Oct 19 00:51:30.529137 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Oct 19 00:51:30.529137 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Oct 19 00:51:30.529137 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Oct 19 00:51:30.529137 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Oct 19 00:51:30.529137 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Oct 19 00:51:30.529137 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Oct 19 00:51:30.529137 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Oct 19 00:51:30.529137 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Oct 19 00:51:30.529137 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Oct 19 00:51:30.529137 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi item = cinderclient( Oct 19 00:51:30.529137 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Oct 19 00:51:30.529137 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Oct 19 00:51:30.529137 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Oct 19 00:51:30.529137 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Oct 19 00:51:30.529137 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Oct 19 00:51:30.529137 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Oct 19 00:51:30.532512 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Oct 19 00:51:30.532512 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Oct 19 00:51:30.532512 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Oct 19 00:51:30.532512 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Oct 19 00:51:30.532512 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume cc2416a6-24da-416f-b1dc-d3d073d178c7 could not be found. Oct 19 00:51:30.532512 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:51:30.532512 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 19 00:51:30.532512 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:51:30.532512 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 19 00:51:30.532512 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 19 00:51:30.532512 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 19 00:51:30.532512 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 19 00:51:30.532512 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 19 00:51:30.532512 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Oct 19 00:51:30.532512 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Oct 19 00:51:30.532512 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 19 00:51:30.532512 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 19 00:51:30.532512 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 122, in show Oct 19 00:51:30.532512 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 19 00:51:30.536118 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume cc2416a6-24da-416f-b1dc-d3d073d178c7 could not be found. Oct 19 00:51:30.536118 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:51:30.536118 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 19 00:51:30.536118 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:51:30.536118 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 19 00:51:30.536118 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 19 00:51:30.536118 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 19 00:51:30.536118 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 19 00:51:30.536118 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 19 00:51:30.536118 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume cc2416a6-24da-416f-b1dc-d3d073d178c7 could not be found. Oct 19 00:51:30.536118 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:51:30.536118 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-45731673-bc7c-4b9c-9f00-d991ede5a19f tempest-VolumesSnapshotsTestJSON-72094637 tempest-VolumesSnapshotsTestJSON-72094637-project] Returning 404 to user: Volume cc2416a6-24da-416f-b1dc-d3d073d178c7 could not be found. {{(pid=125641) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 19 00:51:30.536118 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-45731673-bc7c-4b9c-9f00-d991ede5a19f tempest-VolumesSnapshotsTestJSON-72094637 tempest-VolumesSnapshotsTestJSON-72094637-project] 10.222.0.26 "GET /compute/v2.1/os-volumes/cc2416a6-24da-416f-b1dc-d3d073d178c7" status: 404 len: 109 microversion: 2.1 time: 0.040695 Oct 19 00:51:30.536118 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 903/1018] 10.222.0.26 () {58 vars in 1245 bytes} [Wed Oct 19 00:51:30 2022] GET /compute/v2.1/os-volumes/cc2416a6-24da-416f-b1dc-d3d073d178c7 => generated 109 bytes in 45 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Oct 19 00:51:45.634772 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG dbcounter [-] [125641] Writing DB stats nova_api:SELECT=13 {{(pid=125641) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 00:51:45.636702 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG dbcounter [-] [125641] Writing DB stats nova_cell1:SELECT=80 {{(pid=125641) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 00:51:45.645349 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-e8accd87-e722-441a-887f-828e9c6d2ed9 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:51:45.646050 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-e8accd87-e722-441a-887f-828e9c6d2ed9 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] [instance: ae2afe94-f67b-419f-bd5f-aefc5fbce221] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:51:45.658828 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-e8accd87-e722-441a-887f-828e9c6d2ed9 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:51:45.660483 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-e8accd87-e722-441a-887f-828e9c6d2ed9 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:51:45.661013 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-e8accd87-e722-441a-887f-828e9c6d2ed9 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:51:45.661817 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-e8accd87-e722-441a-887f-828e9c6d2ed9 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:51:45.662388 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-e8accd87-e722-441a-887f-828e9c6d2ed9 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:51:45.663021 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-e8accd87-e722-441a-887f-828e9c6d2ed9 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:51:45.693911 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-e8accd87-e722-441a-887f-828e9c6d2ed9 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] [instance: ae2afe94-f67b-419f-bd5f-aefc5fbce221] Going to try to terminate instance {{(pid=125641) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 19 00:51:45.815417 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-e8accd87-e722-441a-887f-828e9c6d2ed9 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] 10.222.0.26 "DELETE /compute/v2.1/servers/ae2afe94-f67b-419f-bd5f-aefc5fbce221" status: 204 len: 0 microversion: 2.1 time: 0.183107 Oct 19 00:51:45.816677 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 904/1019] 10.222.0.26 () {58 vars in 1239 bytes} [Wed Oct 19 00:51:45 2022] DELETE /compute/v2.1/servers/ae2afe94-f67b-419f-bd5f-aefc5fbce221 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 19 00:51:45.834865 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [req-899cd303-b7e0-41bf-b39e-01f76ad12a67 req-ab47434c-dda0-45b8-ac63-d1c92d73134f service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "ae2afe94-f67b-419f-bd5f-aefc5fbce221", "tag": "c76271cc-707a-4a90-9ae9-135461e8154f"}]} {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 19 00:51:45.847839 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-899cd303-b7e0-41bf-b39e-01f76ad12a67 req-ab47434c-dda0-45b8-ac63-d1c92d73134f service nova] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:51:45.848340 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-899cd303-b7e0-41bf-b39e-01f76ad12a67 req-ab47434c-dda0-45b8-ac63-d1c92d73134f service nova] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:51:45.848839 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-899cd303-b7e0-41bf-b39e-01f76ad12a67 req-ab47434c-dda0-45b8-ac63-d1c92d73134f service nova] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:51:45.862141 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.compute.server_external_events [req-899cd303-b7e0-41bf-b39e-01f76ad12a67 req-ab47434c-dda0-45b8-ac63-d1c92d73134f service nova] Creating event network-changed:c76271cc-707a-4a90-9ae9-135461e8154f for instance ae2afe94-f67b-419f-bd5f-aefc5fbce221 on hv-cinder-maste Oct 19 00:51:45.868275 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [req-899cd303-b7e0-41bf-b39e-01f76ad12a67 req-ab47434c-dda0-45b8-ac63-d1c92d73134f service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.041688 Oct 19 00:51:45.869174 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 905/1020] 10.222.0.26 () {64 vars in 1317 bytes} [Wed Oct 19 00:51:45 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 19 00:51:45.876207 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-e6cccdd5-3e05-4380-a753-84fd36420297 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:51:45.876852 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-e6cccdd5-3e05-4380-a753-84fd36420297 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] [instance: ae2afe94-f67b-419f-bd5f-aefc5fbce221] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:51:45.885252 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-e6cccdd5-3e05-4380-a753-84fd36420297 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:51:45.886914 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-e6cccdd5-3e05-4380-a753-84fd36420297 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:51:45.887758 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-e6cccdd5-3e05-4380-a753-84fd36420297 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:51:45.888708 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-e6cccdd5-3e05-4380-a753-84fd36420297 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:51:45.889352 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-e6cccdd5-3e05-4380-a753-84fd36420297 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:51:45.889949 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-e6cccdd5-3e05-4380-a753-84fd36420297 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:51:46.131314 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-e6cccdd5-3e05-4380-a753-84fd36420297 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f738e67d8c2340028da1254b60d7810e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8684504b0eff415b9063cebae21ca667', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:51:46.148531 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-e6cccdd5-3e05-4380-a753-84fd36420297 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] 10.222.0.26 "GET /compute/v2.1/servers/ae2afe94-f67b-419f-bd5f-aefc5fbce221" status: 200 len: 1576 microversion: 2.1 time: 0.277447 Oct 19 00:51:46.149602 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 906/1021] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:51:45 2022] GET /compute/v2.1/servers/ae2afe94-f67b-419f-bd5f-aefc5fbce221 => generated 1576 bytes in 280 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:51:47.178675 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-16a9ea25-cd32-4512-a9d3-c539d5269a46 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:51:47.181040 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-16a9ea25-cd32-4512-a9d3-c539d5269a46 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] [instance: ae2afe94-f67b-419f-bd5f-aefc5fbce221] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:51:47.193972 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-16a9ea25-cd32-4512-a9d3-c539d5269a46 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:51:47.194817 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-16a9ea25-cd32-4512-a9d3-c539d5269a46 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:51:47.195561 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-16a9ea25-cd32-4512-a9d3-c539d5269a46 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:51:47.196666 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-16a9ea25-cd32-4512-a9d3-c539d5269a46 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:51:47.197268 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-16a9ea25-cd32-4512-a9d3-c539d5269a46 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:51:47.198258 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-16a9ea25-cd32-4512-a9d3-c539d5269a46 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:51:47.445733 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-16a9ea25-cd32-4512-a9d3-c539d5269a46 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f738e67d8c2340028da1254b60d7810e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8684504b0eff415b9063cebae21ca667', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:51:47.456072 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-16a9ea25-cd32-4512-a9d3-c539d5269a46 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] 10.222.0.26 "GET /compute/v2.1/servers/ae2afe94-f67b-419f-bd5f-aefc5fbce221" status: 200 len: 1576 microversion: 2.1 time: 0.285844 Oct 19 00:51:47.456703 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 907/1022] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:51:47 2022] GET /compute/v2.1/servers/ae2afe94-f67b-419f-bd5f-aefc5fbce221 => generated 1576 bytes in 287 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 19 00:51:48.481792 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-40c17694-de8f-4b68-bb8f-ff39a3cfae3a tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:51:48.482910 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-40c17694-de8f-4b68-bb8f-ff39a3cfae3a tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] [instance: ae2afe94-f67b-419f-bd5f-aefc5fbce221] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:51:48.491961 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-40c17694-de8f-4b68-bb8f-ff39a3cfae3a tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:51:48.492608 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-40c17694-de8f-4b68-bb8f-ff39a3cfae3a tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:51:48.493952 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-40c17694-de8f-4b68-bb8f-ff39a3cfae3a tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:51:48.494371 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-40c17694-de8f-4b68-bb8f-ff39a3cfae3a tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:51:48.495583 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-40c17694-de8f-4b68-bb8f-ff39a3cfae3a tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:51:48.496200 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-40c17694-de8f-4b68-bb8f-ff39a3cfae3a tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:51:48.683047 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-40c17694-de8f-4b68-bb8f-ff39a3cfae3a tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f738e67d8c2340028da1254b60d7810e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8684504b0eff415b9063cebae21ca667', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:51:48.694240 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-40c17694-de8f-4b68-bb8f-ff39a3cfae3a tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] 10.222.0.26 "GET /compute/v2.1/servers/ae2afe94-f67b-419f-bd5f-aefc5fbce221" status: 200 len: 1576 microversion: 2.1 time: 0.216731 Oct 19 00:51:48.696592 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 908/1023] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:51:48 2022] GET /compute/v2.1/servers/ae2afe94-f67b-419f-bd5f-aefc5fbce221 => generated 1576 bytes in 220 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 19 00:51:48.965067 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [req-50cbfb51-f67f-4ae7-9f7d-248a86150721 req-6ac3c5a1-59bc-4abb-91db-0fbe981af0a0 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "ae2afe94-f67b-419f-bd5f-aefc5fbce221", "name": "network-vif-deleted", "tag": "c76271cc-707a-4a90-9ae9-135461e8154f"}]} {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 19 00:51:48.982407 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-50cbfb51-f67f-4ae7-9f7d-248a86150721 req-6ac3c5a1-59bc-4abb-91db-0fbe981af0a0 service nova] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:51:48.983651 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-50cbfb51-f67f-4ae7-9f7d-248a86150721 req-6ac3c5a1-59bc-4abb-91db-0fbe981af0a0 service nova] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:51:48.984723 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-50cbfb51-f67f-4ae7-9f7d-248a86150721 req-6ac3c5a1-59bc-4abb-91db-0fbe981af0a0 service nova] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:51:49.006673 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.compute.server_external_events [req-50cbfb51-f67f-4ae7-9f7d-248a86150721 req-6ac3c5a1-59bc-4abb-91db-0fbe981af0a0 service nova] Creating event network-vif-deleted:c76271cc-707a-4a90-9ae9-135461e8154f for instance ae2afe94-f67b-419f-bd5f-aefc5fbce221 on hv-cinder-maste Oct 19 00:51:49.015740 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [req-50cbfb51-f67f-4ae7-9f7d-248a86150721 req-6ac3c5a1-59bc-4abb-91db-0fbe981af0a0 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.057908 Oct 19 00:51:49.016766 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 909/1024] 10.222.0.26 () {64 vars in 1317 bytes} [Wed Oct 19 00:51:48 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 60 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 19 00:51:49.722672 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-ca82e922-2db1-4278-8830-8931966540ce tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:51:49.723807 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-ca82e922-2db1-4278-8830-8931966540ce tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] [instance: ae2afe94-f67b-419f-bd5f-aefc5fbce221] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:51:49.732978 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ca82e922-2db1-4278-8830-8931966540ce tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:51:49.733694 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ca82e922-2db1-4278-8830-8931966540ce tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:51:49.734355 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ca82e922-2db1-4278-8830-8931966540ce tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:51:49.735336 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ca82e922-2db1-4278-8830-8931966540ce tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:51:49.735979 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ca82e922-2db1-4278-8830-8931966540ce tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:51:49.736503 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ca82e922-2db1-4278-8830-8931966540ce tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:51:49.781677 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ca82e922-2db1-4278-8830-8931966540ce tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:51:49.782352 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ca82e922-2db1-4278-8830-8931966540ce tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:51:49.782881 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-ca82e922-2db1-4278-8830-8931966540ce tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:51:49.783410 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.objects.instance [None req-ca82e922-2db1-4278-8830-8931966540ce tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lazy-loading 'fault' on Instance uuid ae2afe94-f67b-419f-bd5f-aefc5fbce221 {{(pid=125641) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 19 00:51:49.835733 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-ca82e922-2db1-4278-8830-8931966540ce tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f738e67d8c2340028da1254b60d7810e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8684504b0eff415b9063cebae21ca667', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:51:49.844554 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-ca82e922-2db1-4278-8830-8931966540ce tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] 10.222.0.26 "GET /compute/v2.1/servers/ae2afe94-f67b-419f-bd5f-aefc5fbce221" status: 200 len: 1230 microversion: 2.1 time: 0.128117 Oct 19 00:51:49.853001 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 910/1025] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:51:49 2022] GET /compute/v2.1/servers/ae2afe94-f67b-419f-bd5f-aefc5fbce221 => generated 1230 bytes in 130 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 19 00:51:50.872840 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-d0d8c1f9-71e5-40c7-967b-8c156ae3c1a1 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:51:50.873649 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-d0d8c1f9-71e5-40c7-967b-8c156ae3c1a1 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] [instance: ae2afe94-f67b-419f-bd5f-aefc5fbce221] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:51:50.880318 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d0d8c1f9-71e5-40c7-967b-8c156ae3c1a1 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:51:50.880810 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d0d8c1f9-71e5-40c7-967b-8c156ae3c1a1 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:51:50.881254 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d0d8c1f9-71e5-40c7-967b-8c156ae3c1a1 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:51:50.881978 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d0d8c1f9-71e5-40c7-967b-8c156ae3c1a1 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:51:50.882438 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d0d8c1f9-71e5-40c7-967b-8c156ae3c1a1 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:51:50.882917 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d0d8c1f9-71e5-40c7-967b-8c156ae3c1a1 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:51:50.898018 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.wsgi [None req-d0d8c1f9-71e5-40c7-967b-8c156ae3c1a1 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] HTTP exception thrown: Instance ae2afe94-f67b-419f-bd5f-aefc5fbce221 could not be found. Oct 19 00:51:50.899751 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-d0d8c1f9-71e5-40c7-967b-8c156ae3c1a1 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Request method failure captured: Oct 19 00:51:50.899751 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: request: GET /compute/v2.1/servers/ae2afe94-f67b-419f-bd5f-aefc5fbce221 HTTP/1.1 Oct 19 00:51:50.899751 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Accept: application/json Oct 19 00:51:50.899751 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Accept-Encoding: identity Oct 19 00:51:50.899751 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Connection: close Oct 19 00:51:50.899751 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Content-Length: 0 Oct 19 00:51:50.899751 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Content-Type: application/json Oct 19 00:51:50.899751 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Host: 10.222.0.26 Oct 19 00:51:50.899751 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Openstack-System-Scope: None Oct 19 00:51:50.899751 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: User-Agent: python-urllib3/1.26.12 Oct 19 00:51:50.899751 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Auth-Token: gAAAAABjT0hR4IKMV3eYr4-XgwodnpQA_lLDwRuR0LL79RvbIfnBA86hBsDcYLHHfVVxdm7zIXuLtp8bh_RiiYg07mmQQk7BvVbyFL3sDfn8L39RJ-FrnzBE24hbvJuW7M5tKiYBZDCP4wGZP3byE26APASrrTWCPUZIjyM-NO7IKZawq3TATyw Oct 19 00:51:50.899751 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Domain-Id: None Oct 19 00:51:50.899751 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Domain-Name: None Oct 19 00:51:50.899751 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Identity-Status: Confirmed Oct 19 00:51:50.899751 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Is-Admin-Project: True Oct 19 00:51:50.899751 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Project-Domain-Id: default Oct 19 00:51:50.899751 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Project-Domain-Name: Default Oct 19 00:51:50.899751 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Project-Id: 8684504b0eff415b9063cebae21ca667 Oct 19 00:51:50.899751 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Project-Name: tempest-TestVolumeBootPattern-1327038966 Oct 19 00:51:50.899751 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Role: member,reader Oct 19 00:51:50.899751 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Roles: member,reader Oct 19 00:51:50.901631 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/8684504b0eff415b9063cebae21ca667"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/8684504b0eff415b9063cebae21ca667"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/8684504b0eff415b9063cebae21ca667"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_8684504b0eff415b9063cebae21ca667"}]}] Oct 19 00:51:50.901631 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Tenant: tempest-TestVolumeBootPattern-1327038966 Oct 19 00:51:50.901631 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Tenant-Id: 8684504b0eff415b9063cebae21ca667 Oct 19 00:51:50.901631 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Tenant-Name: tempest-TestVolumeBootPattern-1327038966 Oct 19 00:51:50.901631 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User: tempest-TestVolumeBootPattern-1327038966-project Oct 19 00:51:50.901631 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User-Domain-Id: default Oct 19 00:51:50.901631 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User-Domain-Name: Default Oct 19 00:51:50.901631 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User-Id: f738e67d8c2340028da1254b60d7810e Oct 19 00:51:50.901631 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User-Name: tempest-TestVolumeBootPattern-1327038966-project Oct 19 00:51:50.901631 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: method: > Oct 19 00:51:50.901631 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: exception: Instance ae2afe94-f67b-419f-bd5f-aefc5fbce221 could not be found. Oct 19 00:51:50.901631 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: action_args: {'id': 'ae2afe94-f67b-419f-bd5f-aefc5fbce221'} Oct 19 00:51:50.902621 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 19 00:51:50.902621 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 19 00:51:50.902621 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 19 00:51:50.902621 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 19 00:51:50.902621 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 19 00:51:50.902621 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 19 00:51:50.902621 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 19 00:51:50.902621 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 19 00:51:50.902621 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 19 00:51:50.902621 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 19 00:51:50.902621 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance ae2afe94-f67b-419f-bd5f-aefc5fbce221 could not be found. Oct 19 00:51:50.902621 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:51:50.902621 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 19 00:51:50.902621 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:51:50.902621 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 19 00:51:50.902621 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 19 00:51:50.902621 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 19 00:51:50.902621 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 19 00:51:50.902621 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 19 00:51:50.902621 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 19 00:51:50.904242 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 19 00:51:50.904242 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 19 00:51:50.904242 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 19 00:51:50.904242 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 19 00:51:50.904242 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 19 00:51:50.904242 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 19 00:51:50.904242 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 19 00:51:50.904242 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance ae2afe94-f67b-419f-bd5f-aefc5fbce221 could not be found. Oct 19 00:51:50.904242 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:51:50.904242 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 19 00:51:50.904242 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:51:50.904242 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 19 00:51:50.904242 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 19 00:51:50.904242 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 19 00:51:50.904242 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 19 00:51:50.904242 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 19 00:51:50.904242 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance ae2afe94-f67b-419f-bd5f-aefc5fbce221 could not be found. Oct 19 00:51:50.904242 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:51:50.905799 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-d0d8c1f9-71e5-40c7-967b-8c156ae3c1a1 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Returning 404 to user: Instance ae2afe94-f67b-419f-bd5f-aefc5fbce221 could not be found. {{(pid=125641) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 19 00:51:50.905799 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-d0d8c1f9-71e5-40c7-967b-8c156ae3c1a1 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] 10.222.0.26 "GET /compute/v2.1/servers/ae2afe94-f67b-419f-bd5f-aefc5fbce221" status: 404 len: 111 microversion: 2.1 time: 0.033159 Oct 19 00:51:50.905799 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 911/1026] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:51:50 2022] GET /compute/v2.1/servers/ae2afe94-f67b-419f-bd5f-aefc5fbce221 => generated 111 bytes in 34 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Oct 19 00:51:54.607139 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-919e98db-55ef-4058-b973-9ad89d55f776 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:51:54.608109 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-919e98db-55ef-4058-b973-9ad89d55f776 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] [instance: d2b07e55-0b24-4055-889f-e69cdf22f251] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:51:54.622610 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-919e98db-55ef-4058-b973-9ad89d55f776 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:51:54.623689 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-919e98db-55ef-4058-b973-9ad89d55f776 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:51:54.624963 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-919e98db-55ef-4058-b973-9ad89d55f776 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:51:54.626525 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-919e98db-55ef-4058-b973-9ad89d55f776 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:51:54.627562 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-919e98db-55ef-4058-b973-9ad89d55f776 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:51:54.628433 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-919e98db-55ef-4058-b973-9ad89d55f776 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:51:54.676395 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-919e98db-55ef-4058-b973-9ad89d55f776 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] [instance: d2b07e55-0b24-4055-889f-e69cdf22f251] Going to try to terminate instance {{(pid=125641) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 19 00:51:54.822305 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-919e98db-55ef-4058-b973-9ad89d55f776 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] 10.222.0.26 "DELETE /compute/v2.1/servers/d2b07e55-0b24-4055-889f-e69cdf22f251" status: 204 len: 0 microversion: 2.1 time: 0.226629 Oct 19 00:51:54.829655 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 912/1027] 10.222.0.26 () {58 vars in 1239 bytes} [Wed Oct 19 00:51:54 2022] DELETE /compute/v2.1/servers/d2b07e55-0b24-4055-889f-e69cdf22f251 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 19 00:51:55.117608 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [req-dae05614-8a32-4c94-8111-d16cdf91279a req-4c8dd2be-a661-462a-b15d-cbe1a5993b92 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "d2b07e55-0b24-4055-889f-e69cdf22f251", "tag": "94f720c6-9b61-4983-ab49-77c9cdd0fe18"}]} {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 19 00:51:55.134672 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-dae05614-8a32-4c94-8111-d16cdf91279a req-4c8dd2be-a661-462a-b15d-cbe1a5993b92 service nova] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:51:55.135387 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-dae05614-8a32-4c94-8111-d16cdf91279a req-4c8dd2be-a661-462a-b15d-cbe1a5993b92 service nova] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:51:55.136137 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-dae05614-8a32-4c94-8111-d16cdf91279a req-4c8dd2be-a661-462a-b15d-cbe1a5993b92 service nova] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:51:55.164384 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.compute.server_external_events [req-dae05614-8a32-4c94-8111-d16cdf91279a req-4c8dd2be-a661-462a-b15d-cbe1a5993b92 service nova] Creating event network-changed:94f720c6-9b61-4983-ab49-77c9cdd0fe18 for instance d2b07e55-0b24-4055-889f-e69cdf22f251 on hv-cinder-maste Oct 19 00:51:55.175521 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [req-dae05614-8a32-4c94-8111-d16cdf91279a req-4c8dd2be-a661-462a-b15d-cbe1a5993b92 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.347762 Oct 19 00:51:55.176627 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 913/1028] 10.222.0.26 () {64 vars in 1317 bytes} [Wed Oct 19 00:51:54 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 351 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 19 00:51:55.187941 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-a1632452-c21c-41fc-bec1-05664a036784 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:51:55.189260 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-a1632452-c21c-41fc-bec1-05664a036784 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] [instance: d2b07e55-0b24-4055-889f-e69cdf22f251] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:51:55.200996 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-a1632452-c21c-41fc-bec1-05664a036784 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:51:55.201862 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-a1632452-c21c-41fc-bec1-05664a036784 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:51:55.202646 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-a1632452-c21c-41fc-bec1-05664a036784 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:51:55.204515 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-a1632452-c21c-41fc-bec1-05664a036784 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:51:55.205462 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-a1632452-c21c-41fc-bec1-05664a036784 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:51:55.206224 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-a1632452-c21c-41fc-bec1-05664a036784 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:51:55.436838 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-a1632452-c21c-41fc-bec1-05664a036784 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f738e67d8c2340028da1254b60d7810e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8684504b0eff415b9063cebae21ca667', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:51:55.445903 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-a1632452-c21c-41fc-bec1-05664a036784 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] 10.222.0.26 "GET /compute/v2.1/servers/d2b07e55-0b24-4055-889f-e69cdf22f251" status: 200 len: 1579 microversion: 2.1 time: 0.267280 Oct 19 00:51:55.449396 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 914/1029] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:51:55 2022] GET /compute/v2.1/servers/d2b07e55-0b24-4055-889f-e69cdf22f251 => generated 1579 bytes in 272 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:51:56.487213 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-bb188413-7728-44b6-a28c-029b4796d3d3 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:51:56.487983 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-bb188413-7728-44b6-a28c-029b4796d3d3 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] [instance: d2b07e55-0b24-4055-889f-e69cdf22f251] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:51:56.495541 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-bb188413-7728-44b6-a28c-029b4796d3d3 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:51:56.496240 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-bb188413-7728-44b6-a28c-029b4796d3d3 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:51:56.496680 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-bb188413-7728-44b6-a28c-029b4796d3d3 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:51:56.497409 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-bb188413-7728-44b6-a28c-029b4796d3d3 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:51:56.498074 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-bb188413-7728-44b6-a28c-029b4796d3d3 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:51:56.498578 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-bb188413-7728-44b6-a28c-029b4796d3d3 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:51:56.668387 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-bb188413-7728-44b6-a28c-029b4796d3d3 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f738e67d8c2340028da1254b60d7810e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8684504b0eff415b9063cebae21ca667', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:51:56.679586 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-bb188413-7728-44b6-a28c-029b4796d3d3 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] 10.222.0.26 "GET /compute/v2.1/servers/d2b07e55-0b24-4055-889f-e69cdf22f251" status: 200 len: 1579 microversion: 2.1 time: 0.196435 Oct 19 00:51:56.694741 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 915/1030] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:51:56 2022] GET /compute/v2.1/servers/d2b07e55-0b24-4055-889f-e69cdf22f251 => generated 1579 bytes in 212 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:51:57.716302 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-1ac25045-1e34-4ec1-9fe4-c0bfd5002aeb tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:51:57.718019 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-1ac25045-1e34-4ec1-9fe4-c0bfd5002aeb tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] [instance: d2b07e55-0b24-4055-889f-e69cdf22f251] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:51:57.730353 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-1ac25045-1e34-4ec1-9fe4-c0bfd5002aeb tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:51:57.731717 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-1ac25045-1e34-4ec1-9fe4-c0bfd5002aeb tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:51:57.732871 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-1ac25045-1e34-4ec1-9fe4-c0bfd5002aeb tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:51:57.734305 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-1ac25045-1e34-4ec1-9fe4-c0bfd5002aeb tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:51:57.735311 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-1ac25045-1e34-4ec1-9fe4-c0bfd5002aeb tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:51:57.736512 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-1ac25045-1e34-4ec1-9fe4-c0bfd5002aeb tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:51:58.001156 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-1ac25045-1e34-4ec1-9fe4-c0bfd5002aeb tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f738e67d8c2340028da1254b60d7810e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8684504b0eff415b9063cebae21ca667', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:51:58.017974 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-1ac25045-1e34-4ec1-9fe4-c0bfd5002aeb tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] 10.222.0.26 "GET /compute/v2.1/servers/d2b07e55-0b24-4055-889f-e69cdf22f251" status: 200 len: 1461 microversion: 2.1 time: 0.311044 Oct 19 00:51:58.019069 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 916/1031] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:51:57 2022] GET /compute/v2.1/servers/d2b07e55-0b24-4055-889f-e69cdf22f251 => generated 1461 bytes in 314 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:51:58.225256 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [req-72b20538-3ba0-4e22-93ad-3595b8399ece req-c0d3c8e9-7ac5-4634-be7b-883fc5409e55 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "d2b07e55-0b24-4055-889f-e69cdf22f251", "name": "network-vif-deleted", "tag": "94f720c6-9b61-4983-ab49-77c9cdd0fe18"}]} {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 19 00:51:58.246407 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-72b20538-3ba0-4e22-93ad-3595b8399ece req-c0d3c8e9-7ac5-4634-be7b-883fc5409e55 service nova] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:51:58.247587 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-72b20538-3ba0-4e22-93ad-3595b8399ece req-c0d3c8e9-7ac5-4634-be7b-883fc5409e55 service nova] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:51:58.248567 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-72b20538-3ba0-4e22-93ad-3595b8399ece req-c0d3c8e9-7ac5-4634-be7b-883fc5409e55 service nova] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:51:58.266284 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.compute.server_external_events [req-72b20538-3ba0-4e22-93ad-3595b8399ece req-c0d3c8e9-7ac5-4634-be7b-883fc5409e55 service nova] Creating event network-vif-deleted:94f720c6-9b61-4983-ab49-77c9cdd0fe18 for instance d2b07e55-0b24-4055-889f-e69cdf22f251 on hv-cinder-maste Oct 19 00:51:58.277055 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [req-72b20538-3ba0-4e22-93ad-3595b8399ece req-c0d3c8e9-7ac5-4634-be7b-883fc5409e55 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.056348 Oct 19 00:51:58.277699 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 917/1032] 10.222.0.26 () {64 vars in 1317 bytes} [Wed Oct 19 00:51:58 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 60 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 19 00:51:59.047566 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-afa96343-73f5-4251-9d82-c2ac82fb26f4 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:51:59.048771 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-afa96343-73f5-4251-9d82-c2ac82fb26f4 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] [instance: d2b07e55-0b24-4055-889f-e69cdf22f251] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:51:59.060290 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-afa96343-73f5-4251-9d82-c2ac82fb26f4 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:51:59.061116 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-afa96343-73f5-4251-9d82-c2ac82fb26f4 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:51:59.061817 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-afa96343-73f5-4251-9d82-c2ac82fb26f4 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:51:59.063071 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-afa96343-73f5-4251-9d82-c2ac82fb26f4 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:51:59.063847 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-afa96343-73f5-4251-9d82-c2ac82fb26f4 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:51:59.064581 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-afa96343-73f5-4251-9d82-c2ac82fb26f4 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:51:59.105140 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-afa96343-73f5-4251-9d82-c2ac82fb26f4 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:51:59.105640 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-afa96343-73f5-4251-9d82-c2ac82fb26f4 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:51:59.106087 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-afa96343-73f5-4251-9d82-c2ac82fb26f4 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:51:59.106580 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.objects.instance [None req-afa96343-73f5-4251-9d82-c2ac82fb26f4 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lazy-loading 'fault' on Instance uuid d2b07e55-0b24-4055-889f-e69cdf22f251 {{(pid=125641) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 19 00:51:59.182628 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-afa96343-73f5-4251-9d82-c2ac82fb26f4 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f738e67d8c2340028da1254b60d7810e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8684504b0eff415b9063cebae21ca667', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:51:59.191291 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-afa96343-73f5-4251-9d82-c2ac82fb26f4 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] 10.222.0.26 "GET /compute/v2.1/servers/d2b07e55-0b24-4055-889f-e69cdf22f251" status: 200 len: 1231 microversion: 2.1 time: 0.151904 Oct 19 00:51:59.192216 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 918/1033] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:51:59 2022] GET /compute/v2.1/servers/d2b07e55-0b24-4055-889f-e69cdf22f251 => generated 1231 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Oct 19 00:52:00.224607 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-d4297007-d13b-4255-a5fa-56e0e8842097 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:52:00.225520 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-d4297007-d13b-4255-a5fa-56e0e8842097 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] [instance: d2b07e55-0b24-4055-889f-e69cdf22f251] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:52:00.234601 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d4297007-d13b-4255-a5fa-56e0e8842097 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:52:00.235270 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d4297007-d13b-4255-a5fa-56e0e8842097 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:52:00.236016 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d4297007-d13b-4255-a5fa-56e0e8842097 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:52:00.237315 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d4297007-d13b-4255-a5fa-56e0e8842097 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:52:00.238019 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d4297007-d13b-4255-a5fa-56e0e8842097 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:52:00.238765 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-d4297007-d13b-4255-a5fa-56e0e8842097 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:52:00.257516 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.wsgi [None req-d4297007-d13b-4255-a5fa-56e0e8842097 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] HTTP exception thrown: Instance d2b07e55-0b24-4055-889f-e69cdf22f251 could not be found. Oct 19 00:52:00.258317 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-d4297007-d13b-4255-a5fa-56e0e8842097 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Request method failure captured: Oct 19 00:52:00.258317 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: request: GET /compute/v2.1/servers/d2b07e55-0b24-4055-889f-e69cdf22f251 HTTP/1.1 Oct 19 00:52:00.258317 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Accept: application/json Oct 19 00:52:00.258317 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Accept-Encoding: identity Oct 19 00:52:00.258317 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Connection: close Oct 19 00:52:00.258317 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Content-Length: 0 Oct 19 00:52:00.258317 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Content-Type: application/json Oct 19 00:52:00.258317 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Host: 10.222.0.26 Oct 19 00:52:00.258317 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Openstack-System-Scope: None Oct 19 00:52:00.258317 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: User-Agent: python-urllib3/1.26.12 Oct 19 00:52:00.258317 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Auth-Token: gAAAAABjT0hR4IKMV3eYr4-XgwodnpQA_lLDwRuR0LL79RvbIfnBA86hBsDcYLHHfVVxdm7zIXuLtp8bh_RiiYg07mmQQk7BvVbyFL3sDfn8L39RJ-FrnzBE24hbvJuW7M5tKiYBZDCP4wGZP3byE26APASrrTWCPUZIjyM-NO7IKZawq3TATyw Oct 19 00:52:00.258317 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Domain-Id: None Oct 19 00:52:00.258317 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Domain-Name: None Oct 19 00:52:00.258317 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Identity-Status: Confirmed Oct 19 00:52:00.258317 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Is-Admin-Project: True Oct 19 00:52:00.258317 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Project-Domain-Id: default Oct 19 00:52:00.258317 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Project-Domain-Name: Default Oct 19 00:52:00.258317 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Project-Id: 8684504b0eff415b9063cebae21ca667 Oct 19 00:52:00.258317 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Project-Name: tempest-TestVolumeBootPattern-1327038966 Oct 19 00:52:00.258317 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Role: member,reader Oct 19 00:52:00.258317 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Roles: member,reader Oct 19 00:52:00.261386 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/8684504b0eff415b9063cebae21ca667"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/8684504b0eff415b9063cebae21ca667"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/8684504b0eff415b9063cebae21ca667"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_8684504b0eff415b9063cebae21ca667"}]}] Oct 19 00:52:00.261386 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Tenant: tempest-TestVolumeBootPattern-1327038966 Oct 19 00:52:00.261386 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Tenant-Id: 8684504b0eff415b9063cebae21ca667 Oct 19 00:52:00.261386 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Tenant-Name: tempest-TestVolumeBootPattern-1327038966 Oct 19 00:52:00.261386 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User: tempest-TestVolumeBootPattern-1327038966-project Oct 19 00:52:00.261386 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User-Domain-Id: default Oct 19 00:52:00.261386 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User-Domain-Name: Default Oct 19 00:52:00.261386 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User-Id: f738e67d8c2340028da1254b60d7810e Oct 19 00:52:00.261386 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User-Name: tempest-TestVolumeBootPattern-1327038966-project Oct 19 00:52:00.261386 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: method: > Oct 19 00:52:00.261386 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: exception: Instance d2b07e55-0b24-4055-889f-e69cdf22f251 could not be found. Oct 19 00:52:00.261386 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: action_args: {'id': 'd2b07e55-0b24-4055-889f-e69cdf22f251'} Oct 19 00:52:00.263660 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 19 00:52:00.263660 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 19 00:52:00.263660 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 19 00:52:00.263660 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 19 00:52:00.263660 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 19 00:52:00.263660 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 19 00:52:00.263660 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 19 00:52:00.263660 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 19 00:52:00.263660 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 19 00:52:00.263660 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 19 00:52:00.263660 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance d2b07e55-0b24-4055-889f-e69cdf22f251 could not be found. Oct 19 00:52:00.263660 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:52:00.263660 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 19 00:52:00.263660 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:52:00.263660 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 19 00:52:00.263660 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 19 00:52:00.263660 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 19 00:52:00.263660 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 19 00:52:00.263660 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 19 00:52:00.263660 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 19 00:52:00.266580 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 19 00:52:00.266580 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 19 00:52:00.266580 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 19 00:52:00.266580 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 19 00:52:00.266580 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 19 00:52:00.266580 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 19 00:52:00.266580 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 19 00:52:00.266580 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance d2b07e55-0b24-4055-889f-e69cdf22f251 could not be found. Oct 19 00:52:00.266580 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:52:00.266580 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 19 00:52:00.266580 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:52:00.266580 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 19 00:52:00.266580 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 19 00:52:00.266580 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 19 00:52:00.266580 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 19 00:52:00.266580 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 19 00:52:00.266580 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance d2b07e55-0b24-4055-889f-e69cdf22f251 could not be found. Oct 19 00:52:00.266580 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:52:00.269373 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-d4297007-d13b-4255-a5fa-56e0e8842097 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Returning 404 to user: Instance d2b07e55-0b24-4055-889f-e69cdf22f251 could not be found. {{(pid=125641) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 19 00:52:00.269373 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-d4297007-d13b-4255-a5fa-56e0e8842097 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] 10.222.0.26 "GET /compute/v2.1/servers/d2b07e55-0b24-4055-889f-e69cdf22f251" status: 404 len: 111 microversion: 2.1 time: 0.040382 Oct 19 00:52:00.269373 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 919/1034] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:52:00 2022] GET /compute/v2.1/servers/d2b07e55-0b24-4055-889f-e69cdf22f251 => generated 111 bytes in 43 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Oct 19 00:52:00.885746 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-f303dc96-a5de-4a49-9f97-bc2f5643470b tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:52:00.886842 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-f303dc96-a5de-4a49-9f97-bc2f5643470b tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] [instance: 0b7c19ee-6d89-4855-bf4b-d35b65bb170b] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:52:00.897000 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-f303dc96-a5de-4a49-9f97-bc2f5643470b tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:52:00.897676 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-f303dc96-a5de-4a49-9f97-bc2f5643470b tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:52:00.898417 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-f303dc96-a5de-4a49-9f97-bc2f5643470b tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:52:00.899446 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-f303dc96-a5de-4a49-9f97-bc2f5643470b tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:52:00.900001 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-f303dc96-a5de-4a49-9f97-bc2f5643470b tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:52:00.901581 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-f303dc96-a5de-4a49-9f97-bc2f5643470b tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:52:00.920236 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.wsgi [None req-f303dc96-a5de-4a49-9f97-bc2f5643470b tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] HTTP exception thrown: Instance 0b7c19ee-6d89-4855-bf4b-d35b65bb170b could not be found. Oct 19 00:52:00.921888 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-f303dc96-a5de-4a49-9f97-bc2f5643470b tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Request method failure captured: Oct 19 00:52:00.921888 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: request: DELETE /compute/v2.1/servers/0b7c19ee-6d89-4855-bf4b-d35b65bb170b HTTP/1.1 Oct 19 00:52:00.921888 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Accept: application/json Oct 19 00:52:00.921888 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Accept-Encoding: identity Oct 19 00:52:00.921888 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Connection: close Oct 19 00:52:00.921888 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Content-Length: 0 Oct 19 00:52:00.921888 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Content-Type: application/json Oct 19 00:52:00.921888 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Host: 10.222.0.26 Oct 19 00:52:00.921888 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Openstack-System-Scope: None Oct 19 00:52:00.921888 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: User-Agent: python-urllib3/1.26.12 Oct 19 00:52:00.921888 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Auth-Token: gAAAAABjT0hR4IKMV3eYr4-XgwodnpQA_lLDwRuR0LL79RvbIfnBA86hBsDcYLHHfVVxdm7zIXuLtp8bh_RiiYg07mmQQk7BvVbyFL3sDfn8L39RJ-FrnzBE24hbvJuW7M5tKiYBZDCP4wGZP3byE26APASrrTWCPUZIjyM-NO7IKZawq3TATyw Oct 19 00:52:00.921888 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Domain-Id: None Oct 19 00:52:00.921888 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Domain-Name: None Oct 19 00:52:00.921888 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Identity-Status: Confirmed Oct 19 00:52:00.921888 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Is-Admin-Project: True Oct 19 00:52:00.921888 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Project-Domain-Id: default Oct 19 00:52:00.921888 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Project-Domain-Name: Default Oct 19 00:52:00.921888 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Project-Id: 8684504b0eff415b9063cebae21ca667 Oct 19 00:52:00.921888 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Project-Name: tempest-TestVolumeBootPattern-1327038966 Oct 19 00:52:00.921888 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Role: member,reader Oct 19 00:52:00.921888 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Roles: member,reader Oct 19 00:52:00.925178 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/8684504b0eff415b9063cebae21ca667"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/8684504b0eff415b9063cebae21ca667"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/8684504b0eff415b9063cebae21ca667"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_8684504b0eff415b9063cebae21ca667"}]}] Oct 19 00:52:00.925178 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Tenant: tempest-TestVolumeBootPattern-1327038966 Oct 19 00:52:00.925178 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Tenant-Id: 8684504b0eff415b9063cebae21ca667 Oct 19 00:52:00.925178 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Tenant-Name: tempest-TestVolumeBootPattern-1327038966 Oct 19 00:52:00.925178 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User: tempest-TestVolumeBootPattern-1327038966-project Oct 19 00:52:00.925178 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User-Domain-Id: default Oct 19 00:52:00.925178 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User-Domain-Name: Default Oct 19 00:52:00.925178 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User-Id: f738e67d8c2340028da1254b60d7810e Oct 19 00:52:00.925178 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User-Name: tempest-TestVolumeBootPattern-1327038966-project Oct 19 00:52:00.925178 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: method: > Oct 19 00:52:00.925178 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: exception: Instance 0b7c19ee-6d89-4855-bf4b-d35b65bb170b could not be found. Oct 19 00:52:00.925178 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: action_args: {'id': '0b7c19ee-6d89-4855-bf4b-d35b65bb170b'} Oct 19 00:52:00.927223 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 19 00:52:00.927223 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 19 00:52:00.927223 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 19 00:52:00.927223 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 19 00:52:00.927223 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 19 00:52:00.927223 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 19 00:52:00.927223 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 19 00:52:00.927223 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 19 00:52:00.927223 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 19 00:52:00.927223 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 19 00:52:00.927223 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 0b7c19ee-6d89-4855-bf4b-d35b65bb170b could not be found. Oct 19 00:52:00.927223 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:52:00.927223 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 19 00:52:00.927223 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:52:00.927223 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 19 00:52:00.927223 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 19 00:52:00.927223 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 19 00:52:00.927223 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 19 00:52:00.927223 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 19 00:52:00.927223 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 19 00:52:00.930867 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 19 00:52:00.930867 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Oct 19 00:52:00.930867 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Oct 19 00:52:00.930867 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Oct 19 00:52:00.930867 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Oct 19 00:52:00.930867 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 19 00:52:00.930867 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 19 00:52:00.930867 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 19 00:52:00.930867 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 19 00:52:00.930867 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 0b7c19ee-6d89-4855-bf4b-d35b65bb170b could not be found. Oct 19 00:52:00.930867 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:52:00.930867 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 19 00:52:00.930867 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:52:00.930867 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 19 00:52:00.930867 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 19 00:52:00.930867 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 19 00:52:00.930867 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 19 00:52:00.930867 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 19 00:52:00.930867 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 0b7c19ee-6d89-4855-bf4b-d35b65bb170b could not be found. Oct 19 00:52:00.930867 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:52:00.935543 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-f303dc96-a5de-4a49-9f97-bc2f5643470b tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Returning 404 to user: Instance 0b7c19ee-6d89-4855-bf4b-d35b65bb170b could not be found. {{(pid=125641) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 19 00:52:00.935543 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-f303dc96-a5de-4a49-9f97-bc2f5643470b tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] 10.222.0.26 "DELETE /compute/v2.1/servers/0b7c19ee-6d89-4855-bf4b-d35b65bb170b" status: 404 len: 111 microversion: 2.1 time: 0.045082 Oct 19 00:52:00.935543 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 920/1035] 10.222.0.26 () {58 vars in 1239 bytes} [Wed Oct 19 00:52:00 2022] DELETE /compute/v2.1/servers/0b7c19ee-6d89-4855-bf4b-d35b65bb170b => generated 111 bytes in 47 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 19 00:52:00.935543 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-bdb7d3bf-20f1-4320-9700-a426ef98f680 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:52:00.935543 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-bdb7d3bf-20f1-4320-9700-a426ef98f680 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] [instance: 0b7c19ee-6d89-4855-bf4b-d35b65bb170b] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:52:00.942303 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-bdb7d3bf-20f1-4320-9700-a426ef98f680 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:52:00.942843 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-bdb7d3bf-20f1-4320-9700-a426ef98f680 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:52:00.943291 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-bdb7d3bf-20f1-4320-9700-a426ef98f680 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:52:00.944204 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-bdb7d3bf-20f1-4320-9700-a426ef98f680 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:52:00.944672 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-bdb7d3bf-20f1-4320-9700-a426ef98f680 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:52:00.945103 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-bdb7d3bf-20f1-4320-9700-a426ef98f680 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:52:00.950383 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG dbcounter [-] [125641] Writing DB stats nova_cell1:SELECT=96,nova_cell1:UPDATE=2,nova_cell1:INSERT=2 {{(pid=125641) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 19 00:52:00.962700 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.wsgi [None req-bdb7d3bf-20f1-4320-9700-a426ef98f680 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] HTTP exception thrown: Instance 0b7c19ee-6d89-4855-bf4b-d35b65bb170b could not be found. Oct 19 00:52:00.965361 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-bdb7d3bf-20f1-4320-9700-a426ef98f680 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Request method failure captured: Oct 19 00:52:00.965361 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: request: GET /compute/v2.1/servers/0b7c19ee-6d89-4855-bf4b-d35b65bb170b HTTP/1.1 Oct 19 00:52:00.965361 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Accept: application/json Oct 19 00:52:00.965361 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Accept-Encoding: identity Oct 19 00:52:00.965361 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Connection: close Oct 19 00:52:00.965361 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Content-Length: 0 Oct 19 00:52:00.965361 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Content-Type: application/json Oct 19 00:52:00.965361 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Host: 10.222.0.26 Oct 19 00:52:00.965361 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Openstack-System-Scope: None Oct 19 00:52:00.965361 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: User-Agent: python-urllib3/1.26.12 Oct 19 00:52:00.965361 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Auth-Token: gAAAAABjT0hR4IKMV3eYr4-XgwodnpQA_lLDwRuR0LL79RvbIfnBA86hBsDcYLHHfVVxdm7zIXuLtp8bh_RiiYg07mmQQk7BvVbyFL3sDfn8L39RJ-FrnzBE24hbvJuW7M5tKiYBZDCP4wGZP3byE26APASrrTWCPUZIjyM-NO7IKZawq3TATyw Oct 19 00:52:00.965361 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Domain-Id: None Oct 19 00:52:00.965361 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Domain-Name: None Oct 19 00:52:00.965361 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Identity-Status: Confirmed Oct 19 00:52:00.965361 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Is-Admin-Project: True Oct 19 00:52:00.965361 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Project-Domain-Id: default Oct 19 00:52:00.965361 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Project-Domain-Name: Default Oct 19 00:52:00.965361 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Project-Id: 8684504b0eff415b9063cebae21ca667 Oct 19 00:52:00.965361 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Project-Name: tempest-TestVolumeBootPattern-1327038966 Oct 19 00:52:00.965361 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Role: member,reader Oct 19 00:52:00.965361 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Roles: member,reader Oct 19 00:52:00.968290 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/8684504b0eff415b9063cebae21ca667"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/8684504b0eff415b9063cebae21ca667"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/8684504b0eff415b9063cebae21ca667"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_8684504b0eff415b9063cebae21ca667"}]}] Oct 19 00:52:00.968290 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Tenant: tempest-TestVolumeBootPattern-1327038966 Oct 19 00:52:00.968290 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Tenant-Id: 8684504b0eff415b9063cebae21ca667 Oct 19 00:52:00.968290 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Tenant-Name: tempest-TestVolumeBootPattern-1327038966 Oct 19 00:52:00.968290 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User: tempest-TestVolumeBootPattern-1327038966-project Oct 19 00:52:00.968290 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User-Domain-Id: default Oct 19 00:52:00.968290 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User-Domain-Name: Default Oct 19 00:52:00.968290 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User-Id: f738e67d8c2340028da1254b60d7810e Oct 19 00:52:00.968290 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User-Name: tempest-TestVolumeBootPattern-1327038966-project Oct 19 00:52:00.968290 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: method: > Oct 19 00:52:00.968290 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: exception: Instance 0b7c19ee-6d89-4855-bf4b-d35b65bb170b could not be found. Oct 19 00:52:00.968290 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: action_args: {'id': '0b7c19ee-6d89-4855-bf4b-d35b65bb170b'} Oct 19 00:52:00.970075 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 19 00:52:00.970075 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 19 00:52:00.970075 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 19 00:52:00.970075 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 19 00:52:00.970075 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 19 00:52:00.970075 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 19 00:52:00.970075 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 19 00:52:00.970075 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 19 00:52:00.970075 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 19 00:52:00.970075 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 19 00:52:00.970075 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 0b7c19ee-6d89-4855-bf4b-d35b65bb170b could not be found. Oct 19 00:52:00.970075 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:52:00.970075 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 19 00:52:00.970075 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:52:00.970075 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 19 00:52:00.970075 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 19 00:52:00.970075 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 19 00:52:00.970075 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 19 00:52:00.970075 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 19 00:52:00.970075 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 19 00:52:00.973619 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 19 00:52:00.973619 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 19 00:52:00.973619 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 19 00:52:00.973619 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 19 00:52:00.973619 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 19 00:52:00.973619 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 19 00:52:00.973619 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 19 00:52:00.973619 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 0b7c19ee-6d89-4855-bf4b-d35b65bb170b could not be found. Oct 19 00:52:00.973619 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:52:00.973619 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 19 00:52:00.973619 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:52:00.973619 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 19 00:52:00.973619 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 19 00:52:00.973619 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 19 00:52:00.973619 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 19 00:52:00.973619 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 19 00:52:00.973619 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 0b7c19ee-6d89-4855-bf4b-d35b65bb170b could not be found. Oct 19 00:52:00.973619 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:52:00.976311 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-bdb7d3bf-20f1-4320-9700-a426ef98f680 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Returning 404 to user: Instance 0b7c19ee-6d89-4855-bf4b-d35b65bb170b could not be found. {{(pid=125641) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 19 00:52:00.976311 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-bdb7d3bf-20f1-4320-9700-a426ef98f680 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] 10.222.0.26 "GET /compute/v2.1/servers/0b7c19ee-6d89-4855-bf4b-d35b65bb170b" status: 404 len: 111 microversion: 2.1 time: 0.037575 Oct 19 00:52:00.976311 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 921/1036] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:52:00 2022] GET /compute/v2.1/servers/0b7c19ee-6d89-4855-bf4b-d35b65bb170b => generated 111 bytes in 42 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Oct 19 00:52:01.336207 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-2b04bcd5-39ca-4974-86d4-6adb3545519a tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Action: 'action', calling method: >, body: {"removeSecurityGroup": {"name": "tempest-secgroup-smoke-1982202811"}} {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 19 00:52:01.337324 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-2b04bcd5-39ca-4974-86d4-6adb3545519a tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] [instance: bfbf8641-41b8-47d8-af4d-ef17c6bd1935] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:52:01.347626 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-2b04bcd5-39ca-4974-86d4-6adb3545519a tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:52:01.348266 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-2b04bcd5-39ca-4974-86d4-6adb3545519a tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:52:01.348799 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-2b04bcd5-39ca-4974-86d4-6adb3545519a tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:52:01.349728 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-2b04bcd5-39ca-4974-86d4-6adb3545519a tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:52:01.350280 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-2b04bcd5-39ca-4974-86d4-6adb3545519a tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:52:01.350865 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-2b04bcd5-39ca-4974-86d4-6adb3545519a tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:52:01.504864 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.network.security_group_api [None req-2b04bcd5-39ca-4974-86d4-6adb3545519a tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Removing security group 871fe458-7962-462f-9d9c-b229a52f3fc4 from port aa62f9af-6447-4516-a59f-dc36fd82da11 Oct 19 00:52:01.950792 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-2b04bcd5-39ca-4974-86d4-6adb3545519a tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] 10.222.0.26 "POST /compute/v2.1/servers/bfbf8641-41b8-47d8-af4d-ef17c6bd1935/action" status: 202 len: 0 microversion: 2.1 time: 0.619189 Oct 19 00:52:01.954994 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 922/1037] 10.222.0.26 () {60 vars in 1278 bytes} [Wed Oct 19 00:52:01 2022] POST /compute/v2.1/servers/bfbf8641-41b8-47d8-af4d-ef17c6bd1935/action => generated 0 bytes in 621 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Oct 19 00:52:01.985200 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [req-ad8b5577-3c7a-4be7-a384-66d3b178b7be req-e6b33e43-1d56-4e72-b6cb-3b6963c27421 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "bfbf8641-41b8-47d8-af4d-ef17c6bd1935", "tag": "aa62f9af-6447-4516-a59f-dc36fd82da11"}]} {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 19 00:52:02.006278 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-ad8b5577-3c7a-4be7-a384-66d3b178b7be req-e6b33e43-1d56-4e72-b6cb-3b6963c27421 service nova] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:52:02.007213 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-ad8b5577-3c7a-4be7-a384-66d3b178b7be req-e6b33e43-1d56-4e72-b6cb-3b6963c27421 service nova] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:52:02.007997 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-ad8b5577-3c7a-4be7-a384-66d3b178b7be req-e6b33e43-1d56-4e72-b6cb-3b6963c27421 service nova] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:52:02.027086 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.compute.server_external_events [req-ad8b5577-3c7a-4be7-a384-66d3b178b7be req-e6b33e43-1d56-4e72-b6cb-3b6963c27421 service nova] Creating event network-changed:aa62f9af-6447-4516-a59f-dc36fd82da11 for instance bfbf8641-41b8-47d8-af4d-ef17c6bd1935 on hv-cinder-maste Oct 19 00:52:02.032889 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [req-ad8b5577-3c7a-4be7-a384-66d3b178b7be req-e6b33e43-1d56-4e72-b6cb-3b6963c27421 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.057789 Oct 19 00:52:02.034106 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 923/1038] 10.222.0.26 () {64 vars in 1317 bytes} [Wed Oct 19 00:52:01 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 60 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 19 00:52:02.544325 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-7c2ceec3-5588-46be-a2d8-250b150e4465 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] Calling method '.version_select at 0x7f71c388e430>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:52:02.580414 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-7c2ceec3-5588-46be-a2d8-250b150e4465 tempest-TestVolumeBootPattern-1327038966 tempest-TestVolumeBootPattern-1327038966-project] 10.222.0.26 "DELETE /compute/v2.1/os-keypairs/tempest-TestVolumeBootPattern-1204843348" status: 202 len: 0 microversion: 2.1 time: 0.041544 Oct 19 00:52:02.584078 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 924/1039] 10.222.0.26 () {58 vars in 1263 bytes} [Wed Oct 19 00:52:02 2022] DELETE /compute/v2.1/os-keypairs/tempest-TestVolumeBootPattern-1204843348 => generated 0 bytes in 46 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 19 00:52:03.926664 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-fc5a650d-624e-4905-b669-e52deffaa426 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:52:03.927854 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-fc5a650d-624e-4905-b669-e52deffaa426 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] [instance: bfbf8641-41b8-47d8-af4d-ef17c6bd1935] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:52:03.939398 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-fc5a650d-624e-4905-b669-e52deffaa426 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:52:03.940314 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-fc5a650d-624e-4905-b669-e52deffaa426 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:52:03.941089 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-fc5a650d-624e-4905-b669-e52deffaa426 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:52:03.942738 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-fc5a650d-624e-4905-b669-e52deffaa426 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:52:03.943495 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-fc5a650d-624e-4905-b669-e52deffaa426 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:52:03.944238 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-fc5a650d-624e-4905-b669-e52deffaa426 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:52:04.079060 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.objects.instance [None req-fc5a650d-624e-4905-b669-e52deffaa426 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Lazy-loading 'services' on Instance uuid bfbf8641-41b8-47d8-af4d-ef17c6bd1935 {{(pid=125641) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 19 00:52:04.256045 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-fc5a650d-624e-4905-b669-e52deffaa426 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] 10.222.0.26 "DELETE /compute/v2.1/servers/bfbf8641-41b8-47d8-af4d-ef17c6bd1935/os-volume_attachments/b9d30ae7-badc-4163-827a-140033caedf8" status: 202 len: 0 microversion: 2.1 time: 0.333685 Oct 19 00:52:04.257622 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 925/1040] 10.222.0.26 () {58 vars in 1416 bytes} [Wed Oct 19 00:52:03 2022] DELETE /compute/v2.1/servers/bfbf8641-41b8-47d8-af4d-ef17c6bd1935/os-volume_attachments/b9d30ae7-badc-4163-827a-140033caedf8 => generated 0 bytes in 338 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Oct 19 00:52:04.285954 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [req-55db4d65-c8f9-4449-bb0c-3ac26f2c031f req-f55efc9b-e682-4d56-af4f-dc58b86484ee service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "bfbf8641-41b8-47d8-af4d-ef17c6bd1935", "tag": "aa62f9af-6447-4516-a59f-dc36fd82da11"}]} {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 19 00:52:04.321566 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-55db4d65-c8f9-4449-bb0c-3ac26f2c031f req-f55efc9b-e682-4d56-af4f-dc58b86484ee service nova] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:52:04.323399 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-55db4d65-c8f9-4449-bb0c-3ac26f2c031f req-f55efc9b-e682-4d56-af4f-dc58b86484ee service nova] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:52:04.325198 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-55db4d65-c8f9-4449-bb0c-3ac26f2c031f req-f55efc9b-e682-4d56-af4f-dc58b86484ee service nova] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:52:04.348793 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.compute.server_external_events [req-55db4d65-c8f9-4449-bb0c-3ac26f2c031f req-f55efc9b-e682-4d56-af4f-dc58b86484ee service nova] Creating event network-changed:aa62f9af-6447-4516-a59f-dc36fd82da11 for instance bfbf8641-41b8-47d8-af4d-ef17c6bd1935 on hv-cinder-maste Oct 19 00:52:04.359400 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [req-55db4d65-c8f9-4449-bb0c-3ac26f2c031f req-f55efc9b-e682-4d56-af4f-dc58b86484ee service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.098779 Oct 19 00:52:04.362066 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 926/1041] 10.222.0.26 () {64 vars in 1317 bytes} [Wed Oct 19 00:52:04 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 19 00:52:04.381215 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-1a3bf406-05f8-4487-a030-df7c00a71fba tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:52:04.387923 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-1a3bf406-05f8-4487-a030-df7c00a71fba tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] [instance: bfbf8641-41b8-47d8-af4d-ef17c6bd1935] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:52:04.409011 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-1a3bf406-05f8-4487-a030-df7c00a71fba tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:52:04.410745 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-1a3bf406-05f8-4487-a030-df7c00a71fba tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:52:04.412603 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-1a3bf406-05f8-4487-a030-df7c00a71fba tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:52:04.421597 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-1a3bf406-05f8-4487-a030-df7c00a71fba tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:52:04.423704 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-1a3bf406-05f8-4487-a030-df7c00a71fba tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:52:04.424931 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-1a3bf406-05f8-4487-a030-df7c00a71fba tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:52:04.476596 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-1a3bf406-05f8-4487-a030-df7c00a71fba tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] 10.222.0.26 "GET /compute/v2.1/servers/bfbf8641-41b8-47d8-af4d-ef17c6bd1935/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.107704 Oct 19 00:52:04.483169 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 927/1042] 10.222.0.26 () {58 vars in 1302 bytes} [Wed Oct 19 00:52:04 2022] GET /compute/v2.1/servers/bfbf8641-41b8-47d8-af4d-ef17c6bd1935/os-volume_attachments => generated 197 bytes in 120 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Oct 19 00:52:05.490656 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-7d46655b-68b1-4a01-9935-4cf1f76b4eb5 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:52:05.496087 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-7d46655b-68b1-4a01-9935-4cf1f76b4eb5 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] [instance: bfbf8641-41b8-47d8-af4d-ef17c6bd1935] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:52:05.512649 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-7d46655b-68b1-4a01-9935-4cf1f76b4eb5 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:52:05.514208 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-7d46655b-68b1-4a01-9935-4cf1f76b4eb5 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:52:05.516350 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-7d46655b-68b1-4a01-9935-4cf1f76b4eb5 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:52:05.519534 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-7d46655b-68b1-4a01-9935-4cf1f76b4eb5 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:52:05.520680 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-7d46655b-68b1-4a01-9935-4cf1f76b4eb5 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:52:05.521734 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-7d46655b-68b1-4a01-9935-4cf1f76b4eb5 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:52:05.612390 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-7d46655b-68b1-4a01-9935-4cf1f76b4eb5 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] 10.222.0.26 "GET /compute/v2.1/servers/bfbf8641-41b8-47d8-af4d-ef17c6bd1935/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.126152 Oct 19 00:52:05.621648 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 928/1043] 10.222.0.26 () {58 vars in 1302 bytes} [Wed Oct 19 00:52:05 2022] GET /compute/v2.1/servers/bfbf8641-41b8-47d8-af4d-ef17c6bd1935/os-volume_attachments => generated 197 bytes in 136 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 19 00:52:06.628906 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-1185dc8e-124e-47a1-b776-3713461c08b6 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:52:06.632529 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-1185dc8e-124e-47a1-b776-3713461c08b6 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] [instance: bfbf8641-41b8-47d8-af4d-ef17c6bd1935] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:52:06.641712 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-1185dc8e-124e-47a1-b776-3713461c08b6 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:52:06.642804 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-1185dc8e-124e-47a1-b776-3713461c08b6 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:52:06.643512 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-1185dc8e-124e-47a1-b776-3713461c08b6 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:52:06.645546 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-1185dc8e-124e-47a1-b776-3713461c08b6 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:52:06.646203 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-1185dc8e-124e-47a1-b776-3713461c08b6 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:52:06.646887 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-1185dc8e-124e-47a1-b776-3713461c08b6 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:52:06.708018 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-1185dc8e-124e-47a1-b776-3713461c08b6 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] 10.222.0.26 "GET /compute/v2.1/servers/bfbf8641-41b8-47d8-af4d-ef17c6bd1935/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.082819 Oct 19 00:52:06.719129 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 929/1044] 10.222.0.26 () {58 vars in 1302 bytes} [Wed Oct 19 00:52:06 2022] GET /compute/v2.1/servers/bfbf8641-41b8-47d8-af4d-ef17c6bd1935/os-volume_attachments => generated 25 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Oct 19 00:52:06.724535 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-58ffed3d-0e0c-4ea4-84bd-1be60569ea26 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:52:06.725184 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-58ffed3d-0e0c-4ea4-84bd-1be60569ea26 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] [instance: bfbf8641-41b8-47d8-af4d-ef17c6bd1935] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:52:06.734354 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-58ffed3d-0e0c-4ea4-84bd-1be60569ea26 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:52:06.735159 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-58ffed3d-0e0c-4ea4-84bd-1be60569ea26 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:52:06.735745 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-58ffed3d-0e0c-4ea4-84bd-1be60569ea26 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:52:06.736889 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-58ffed3d-0e0c-4ea4-84bd-1be60569ea26 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:52:06.737643 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-58ffed3d-0e0c-4ea4-84bd-1be60569ea26 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:52:06.738221 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-58ffed3d-0e0c-4ea4-84bd-1be60569ea26 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:52:06.908185 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.wsgi [None req-58ffed3d-0e0c-4ea4-84bd-1be60569ea26 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] HTTP exception thrown: Instance bfbf8641-41b8-47d8-af4d-ef17c6bd1935 is not attached to volume b9d30ae7-badc-4163-827a-140033caedf8 Oct 19 00:52:06.911578 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-58ffed3d-0e0c-4ea4-84bd-1be60569ea26 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Request method failure captured: Oct 19 00:52:06.911578 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: request: DELETE /compute/v2.1/servers/bfbf8641-41b8-47d8-af4d-ef17c6bd1935/os-volume_attachments/b9d30ae7-badc-4163-827a-140033caedf8 HTTP/1.1 Oct 19 00:52:06.911578 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Accept: application/json Oct 19 00:52:06.911578 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Accept-Encoding: identity Oct 19 00:52:06.911578 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Connection: close Oct 19 00:52:06.911578 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Content-Length: 0 Oct 19 00:52:06.911578 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Content-Type: application/json Oct 19 00:52:06.911578 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Host: 10.222.0.26 Oct 19 00:52:06.911578 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Openstack-System-Scope: None Oct 19 00:52:06.911578 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: User-Agent: python-urllib3/1.26.12 Oct 19 00:52:06.911578 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Auth-Token: gAAAAABjT0mv8KQHQalU07zbcIYjP8VpJ4xW9cLV04aB90DyvJs2eKcaPvac899AnYNmV-oCzZg8UkG5lplNW8-z7nGb5tvDwbR9dA3avIr2xWDBUfdRha1J0QVOLKtrXYQ8BnKlkccTe2p5Wer0TgNuQNLeVI-mZGwtsFjELvrR4h6dysRsW4g Oct 19 00:52:06.911578 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Domain-Id: None Oct 19 00:52:06.911578 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Domain-Name: None Oct 19 00:52:06.911578 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Identity-Status: Confirmed Oct 19 00:52:06.911578 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Is-Admin-Project: True Oct 19 00:52:06.911578 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Project-Domain-Id: default Oct 19 00:52:06.911578 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Project-Domain-Name: Default Oct 19 00:52:06.911578 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Project-Id: a8a6d900d9d344a694b955f51fa46aa9 Oct 19 00:52:06.911578 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Project-Name: tempest-TestMinimumBasicScenario-386785066 Oct 19 00:52:06.911578 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Role: member,reader Oct 19 00:52:06.911578 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Roles: member,reader Oct 19 00:52:06.913415 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/a8a6d900d9d344a694b955f51fa46aa9"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/a8a6d900d9d344a694b955f51fa46aa9"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/a8a6d900d9d344a694b955f51fa46aa9"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_a8a6d900d9d344a694b955f51fa46aa9"}]}] Oct 19 00:52:06.913415 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Tenant: tempest-TestMinimumBasicScenario-386785066 Oct 19 00:52:06.913415 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Tenant-Id: a8a6d900d9d344a694b955f51fa46aa9 Oct 19 00:52:06.913415 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Tenant-Name: tempest-TestMinimumBasicScenario-386785066 Oct 19 00:52:06.913415 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User: tempest-TestMinimumBasicScenario-386785066-project Oct 19 00:52:06.913415 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User-Domain-Id: default Oct 19 00:52:06.913415 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User-Domain-Name: Default Oct 19 00:52:06.913415 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User-Id: 531efd120096452caf8c0b7a7f6b9ba1 Oct 19 00:52:06.913415 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User-Name: tempest-TestMinimumBasicScenario-386785066-project Oct 19 00:52:06.913415 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: method: > Oct 19 00:52:06.913415 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: exception: Instance bfbf8641-41b8-47d8-af4d-ef17c6bd1935 is not attached to volume b9d30ae7-badc-4163-827a-140033caedf8 Oct 19 00:52:06.914376 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: action_args: {'server_id': 'bfbf8641-41b8-47d8-af4d-ef17c6bd1935', 'id': 'b9d30ae7-badc-4163-827a-140033caedf8'} Oct 19 00:52:06.914376 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 19 00:52:06.914376 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 19 00:52:06.914376 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 552, in delete Oct 19 00:52:06.914376 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi bdm = objects.BlockDeviceMapping.get_by_volume_and_instance( Oct 19 00:52:06.914376 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Oct 19 00:52:06.914376 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Oct 19 00:52:06.914376 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/block_device.py", line 293, in get_by_volume_and_instance Oct 19 00:52:06.914376 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi raise exception.VolumeBDMNotFound(volume_id=volume_id) Oct 19 00:52:06.914376 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi nova.exception.VolumeBDMNotFound: No volume Block Device Mapping with id b9d30ae7-badc-4163-827a-140033caedf8. Oct 19 00:52:06.914376 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:52:06.914376 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 19 00:52:06.914376 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:52:06.914376 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 19 00:52:06.914376 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 19 00:52:06.914376 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 19 00:52:06.914376 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 19 00:52:06.914376 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 19 00:52:06.914376 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 19 00:52:06.916081 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 19 00:52:06.916081 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 558, in delete Oct 19 00:52:06.916081 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=msg) Oct 19 00:52:06.916081 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance bfbf8641-41b8-47d8-af4d-ef17c6bd1935 is not attached to volume b9d30ae7-badc-4163-827a-140033caedf8 Oct 19 00:52:06.916081 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:52:06.916081 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 19 00:52:06.916081 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:52:06.916081 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 19 00:52:06.916081 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 19 00:52:06.916081 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 19 00:52:06.916081 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 19 00:52:06.916081 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 19 00:52:06.916081 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance bfbf8641-41b8-47d8-af4d-ef17c6bd1935 is not attached to volume b9d30ae7-badc-4163-827a-140033caedf8 Oct 19 00:52:06.916081 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:52:06.918543 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-58ffed3d-0e0c-4ea4-84bd-1be60569ea26 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Returning 404 to user: Instance bfbf8641-41b8-47d8-af4d-ef17c6bd1935 is not attached to volume b9d30ae7-badc-4163-827a-140033caedf8 {{(pid=125641) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 19 00:52:06.920306 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-58ffed3d-0e0c-4ea4-84bd-1be60569ea26 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] 10.222.0.26 "DELETE /compute/v2.1/servers/bfbf8641-41b8-47d8-af4d-ef17c6bd1935/os-volume_attachments/b9d30ae7-badc-4163-827a-140033caedf8" status: 404 len: 154 microversion: 2.1 time: 0.199926 Oct 19 00:52:06.921567 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 930/1045] 10.222.0.26 () {58 vars in 1416 bytes} [Wed Oct 19 00:52:06 2022] DELETE /compute/v2.1/servers/bfbf8641-41b8-47d8-af4d-ef17c6bd1935/os-volume_attachments/b9d30ae7-badc-4163-827a-140033caedf8 => generated 154 bytes in 202 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Oct 19 00:52:08.152958 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-f9393200-45b9-42bd-8a45-24dd4accda28 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:52:08.155341 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-f9393200-45b9-42bd-8a45-24dd4accda28 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] [instance: bfbf8641-41b8-47d8-af4d-ef17c6bd1935] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:52:08.175586 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-f9393200-45b9-42bd-8a45-24dd4accda28 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:52:08.176658 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-f9393200-45b9-42bd-8a45-24dd4accda28 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:52:08.177492 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-f9393200-45b9-42bd-8a45-24dd4accda28 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:52:08.180166 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-f9393200-45b9-42bd-8a45-24dd4accda28 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:52:08.181102 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-f9393200-45b9-42bd-8a45-24dd4accda28 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:52:08.182774 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-f9393200-45b9-42bd-8a45-24dd4accda28 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:52:08.249562 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-f9393200-45b9-42bd-8a45-24dd4accda28 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] [instance: bfbf8641-41b8-47d8-af4d-ef17c6bd1935] Going to try to terminate instance {{(pid=125641) delete /opt/stack/nova/nova/compute/api.py:2719}} Oct 19 00:52:08.468951 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-f9393200-45b9-42bd-8a45-24dd4accda28 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] 10.222.0.26 "DELETE /compute/v2.1/servers/bfbf8641-41b8-47d8-af4d-ef17c6bd1935" status: 204 len: 0 microversion: 2.1 time: 0.324207 Oct 19 00:52:08.470182 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 931/1046] 10.222.0.26 () {58 vars in 1239 bytes} [Wed Oct 19 00:52:08 2022] DELETE /compute/v2.1/servers/bfbf8641-41b8-47d8-af4d-ef17c6bd1935 => generated 0 bytes in 328 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Oct 19 00:52:08.483730 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-53c11263-b8e0-4a6c-b2d2-f77ab619c1ed tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:52:08.484875 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-53c11263-b8e0-4a6c-b2d2-f77ab619c1ed tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] [instance: bfbf8641-41b8-47d8-af4d-ef17c6bd1935] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:52:08.491893 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-53c11263-b8e0-4a6c-b2d2-f77ab619c1ed tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:52:08.492399 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-53c11263-b8e0-4a6c-b2d2-f77ab619c1ed tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:52:08.492823 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-53c11263-b8e0-4a6c-b2d2-f77ab619c1ed tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:52:08.493628 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-53c11263-b8e0-4a6c-b2d2-f77ab619c1ed tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:52:08.495291 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-53c11263-b8e0-4a6c-b2d2-f77ab619c1ed tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:52:08.496181 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-53c11263-b8e0-4a6c-b2d2-f77ab619c1ed tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:52:08.709081 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-53c11263-b8e0-4a6c-b2d2-f77ab619c1ed tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '531efd120096452caf8c0b7a7f6b9ba1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a8a6d900d9d344a694b955f51fa46aa9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:52:08.724757 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-53c11263-b8e0-4a6c-b2d2-f77ab619c1ed tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] 10.222.0.26 "GET /compute/v2.1/servers/bfbf8641-41b8-47d8-af4d-ef17c6bd1935" status: 200 len: 1554 microversion: 2.1 time: 0.246687 Oct 19 00:52:08.725960 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 932/1047] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:52:08 2022] GET /compute/v2.1/servers/bfbf8641-41b8-47d8-af4d-ef17c6bd1935 => generated 1554 bytes in 250 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 19 00:52:09.754017 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-810a3e7a-a4cb-4fa9-9553-a9df2bc71aa7 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:52:09.755372 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-810a3e7a-a4cb-4fa9-9553-a9df2bc71aa7 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] [instance: bfbf8641-41b8-47d8-af4d-ef17c6bd1935] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:52:09.768406 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-810a3e7a-a4cb-4fa9-9553-a9df2bc71aa7 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:52:09.769434 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-810a3e7a-a4cb-4fa9-9553-a9df2bc71aa7 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:52:09.770356 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-810a3e7a-a4cb-4fa9-9553-a9df2bc71aa7 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:52:09.771846 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-810a3e7a-a4cb-4fa9-9553-a9df2bc71aa7 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:52:09.773229 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-810a3e7a-a4cb-4fa9-9553-a9df2bc71aa7 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:52:09.774095 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-810a3e7a-a4cb-4fa9-9553-a9df2bc71aa7 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:52:09.996807 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-810a3e7a-a4cb-4fa9-9553-a9df2bc71aa7 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '531efd120096452caf8c0b7a7f6b9ba1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a8a6d900d9d344a694b955f51fa46aa9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:52:10.017359 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-810a3e7a-a4cb-4fa9-9553-a9df2bc71aa7 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] 10.222.0.26 "GET /compute/v2.1/servers/bfbf8641-41b8-47d8-af4d-ef17c6bd1935" status: 200 len: 1554 microversion: 2.1 time: 0.270152 Oct 19 00:52:10.019241 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 933/1048] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:52:09 2022] GET /compute/v2.1/servers/bfbf8641-41b8-47d8-af4d-ef17c6bd1935 => generated 1554 bytes in 274 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 19 00:52:11.068565 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-e624b576-88de-4e1c-a130-c03af0000d6a tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:52:11.069495 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-e624b576-88de-4e1c-a130-c03af0000d6a tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] [instance: bfbf8641-41b8-47d8-af4d-ef17c6bd1935] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:52:11.077050 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-e624b576-88de-4e1c-a130-c03af0000d6a tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:52:11.077650 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-e624b576-88de-4e1c-a130-c03af0000d6a tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:52:11.078163 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-e624b576-88de-4e1c-a130-c03af0000d6a tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:52:11.078979 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-e624b576-88de-4e1c-a130-c03af0000d6a tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:52:11.079579 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-e624b576-88de-4e1c-a130-c03af0000d6a tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:52:11.080003 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-e624b576-88de-4e1c-a130-c03af0000d6a tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:52:11.291575 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-e624b576-88de-4e1c-a130-c03af0000d6a tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '531efd120096452caf8c0b7a7f6b9ba1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a8a6d900d9d344a694b955f51fa46aa9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:52:11.311966 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-e624b576-88de-4e1c-a130-c03af0000d6a tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] 10.222.0.26 "GET /compute/v2.1/servers/bfbf8641-41b8-47d8-af4d-ef17c6bd1935" status: 200 len: 1554 microversion: 2.1 time: 0.247308 Oct 19 00:52:11.318682 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 934/1049] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:52:11 2022] GET /compute/v2.1/servers/bfbf8641-41b8-47d8-af4d-ef17c6bd1935 => generated 1554 bytes in 251 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 19 00:52:11.928212 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [req-7cdc16a0-e226-43e2-8032-80ec794d42a2 req-6984f637-799d-4474-ba71-24f9cef42d85 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "bfbf8641-41b8-47d8-af4d-ef17c6bd1935", "name": "network-vif-deleted", "tag": "aa62f9af-6447-4516-a59f-dc36fd82da11"}]} {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Oct 19 00:52:11.959291 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-7cdc16a0-e226-43e2-8032-80ec794d42a2 req-6984f637-799d-4474-ba71-24f9cef42d85 service nova] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:52:11.964026 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-7cdc16a0-e226-43e2-8032-80ec794d42a2 req-6984f637-799d-4474-ba71-24f9cef42d85 service nova] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:52:11.970745 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [req-7cdc16a0-e226-43e2-8032-80ec794d42a2 req-6984f637-799d-4474-ba71-24f9cef42d85 service nova] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:52:11.999286 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.compute.server_external_events [req-7cdc16a0-e226-43e2-8032-80ec794d42a2 req-6984f637-799d-4474-ba71-24f9cef42d85 service nova] Creating event network-vif-deleted:aa62f9af-6447-4516-a59f-dc36fd82da11 for instance bfbf8641-41b8-47d8-af4d-ef17c6bd1935 on hv-cinder-maste Oct 19 00:52:12.009385 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [req-7cdc16a0-e226-43e2-8032-80ec794d42a2 req-6984f637-799d-4474-ba71-24f9cef42d85 service nova] 10.222.0.26 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.091207 Oct 19 00:52:12.010565 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 935/1050] 10.222.0.26 () {64 vars in 1317 bytes} [Wed Oct 19 00:52:11 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Oct 19 00:52:12.337165 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-b97a21c4-bdb1-4105-b3f6-9e0aba5d79bf tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:52:12.339782 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-b97a21c4-bdb1-4105-b3f6-9e0aba5d79bf tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] [instance: bfbf8641-41b8-47d8-af4d-ef17c6bd1935] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:52:12.349772 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-b97a21c4-bdb1-4105-b3f6-9e0aba5d79bf tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:52:12.350852 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-b97a21c4-bdb1-4105-b3f6-9e0aba5d79bf tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:52:12.351647 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-b97a21c4-bdb1-4105-b3f6-9e0aba5d79bf tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:52:12.352572 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-b97a21c4-bdb1-4105-b3f6-9e0aba5d79bf tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:52:12.354115 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-b97a21c4-bdb1-4105-b3f6-9e0aba5d79bf tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:52:12.355016 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-b97a21c4-bdb1-4105-b3f6-9e0aba5d79bf tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:52:12.405778 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-b97a21c4-bdb1-4105-b3f6-9e0aba5d79bf tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:52:12.406740 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-b97a21c4-bdb1-4105-b3f6-9e0aba5d79bf tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:52:12.407428 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-b97a21c4-bdb1-4105-b3f6-9e0aba5d79bf tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:52:12.408129 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.objects.instance [None req-b97a21c4-bdb1-4105-b3f6-9e0aba5d79bf tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Lazy-loading 'fault' on Instance uuid bfbf8641-41b8-47d8-af4d-ef17c6bd1935 {{(pid=125641) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Oct 19 00:52:12.459600 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.policy [None req-b97a21c4-bdb1-4105-b3f6-9e0aba5d79bf tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '531efd120096452caf8c0b7a7f6b9ba1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a8a6d900d9d344a694b955f51fa46aa9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=125641) authorize /opt/stack/nova/nova/policy.py:199}} Oct 19 00:52:12.480753 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-b97a21c4-bdb1-4105-b3f6-9e0aba5d79bf tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] 10.222.0.26 "GET /compute/v2.1/servers/bfbf8641-41b8-47d8-af4d-ef17c6bd1935" status: 200 len: 1348 microversion: 2.1 time: 0.147816 Oct 19 00:52:12.491364 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 936/1051] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:52:12 2022] GET /compute/v2.1/servers/bfbf8641-41b8-47d8-af4d-ef17c6bd1935 => generated 1348 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Oct 19 00:52:13.526234 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-b5b33a43-a3ec-432e-8265-208c3eef3b89 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Calling method '>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:52:13.528670 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.compute.api [None req-b5b33a43-a3ec-432e-8265-208c3eef3b89 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] [instance: bfbf8641-41b8-47d8-af4d-ef17c6bd1935] Fetching instance by UUID {{(pid=125641) get /opt/stack/nova/nova/compute/api.py:2975}} Oct 19 00:52:13.544593 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-b5b33a43-a3ec-432e-8265-208c3eef3b89 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:52:13.545731 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-b5b33a43-a3ec-432e-8265-208c3eef3b89 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:52:13.547311 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-b5b33a43-a3ec-432e-8265-208c3eef3b89 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:52:13.548933 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-b5b33a43-a3ec-432e-8265-208c3eef3b89 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Acquiring lock "3d98fd57-b040-4364-b7a0-5303a14589e8" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Oct 19 00:52:13.550515 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-b5b33a43-a3ec-432e-8265-208c3eef3b89 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Oct 19 00:52:13.551739 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG oslo_concurrency.lockutils [None req-b5b33a43-a3ec-432e-8265-208c3eef3b89 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Lock "3d98fd57-b040-4364-b7a0-5303a14589e8" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=125641) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Oct 19 00:52:13.596304 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.wsgi [None req-b5b33a43-a3ec-432e-8265-208c3eef3b89 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] HTTP exception thrown: Instance bfbf8641-41b8-47d8-af4d-ef17c6bd1935 could not be found. Oct 19 00:52:13.599140 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-b5b33a43-a3ec-432e-8265-208c3eef3b89 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Request method failure captured: Oct 19 00:52:13.599140 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: request: GET /compute/v2.1/servers/bfbf8641-41b8-47d8-af4d-ef17c6bd1935 HTTP/1.1 Oct 19 00:52:13.599140 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Accept: application/json Oct 19 00:52:13.599140 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Accept-Encoding: identity Oct 19 00:52:13.599140 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Connection: close Oct 19 00:52:13.599140 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Content-Length: 0 Oct 19 00:52:13.599140 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Content-Type: application/json Oct 19 00:52:13.599140 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Host: 10.222.0.26 Oct 19 00:52:13.599140 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: Openstack-System-Scope: None Oct 19 00:52:13.599140 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: User-Agent: python-urllib3/1.26.12 Oct 19 00:52:13.599140 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Auth-Token: gAAAAABjT0mv8KQHQalU07zbcIYjP8VpJ4xW9cLV04aB90DyvJs2eKcaPvac899AnYNmV-oCzZg8UkG5lplNW8-z7nGb5tvDwbR9dA3avIr2xWDBUfdRha1J0QVOLKtrXYQ8BnKlkccTe2p5Wer0TgNuQNLeVI-mZGwtsFjELvrR4h6dysRsW4g Oct 19 00:52:13.599140 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Domain-Id: None Oct 19 00:52:13.599140 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Domain-Name: None Oct 19 00:52:13.599140 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Identity-Status: Confirmed Oct 19 00:52:13.599140 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Is-Admin-Project: True Oct 19 00:52:13.599140 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Project-Domain-Id: default Oct 19 00:52:13.599140 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Project-Domain-Name: Default Oct 19 00:52:13.599140 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Project-Id: a8a6d900d9d344a694b955f51fa46aa9 Oct 19 00:52:13.599140 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Project-Name: tempest-TestMinimumBasicScenario-386785066 Oct 19 00:52:13.599140 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Role: member,reader Oct 19 00:52:13.599140 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Roles: member,reader Oct 19 00:52:13.602180 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Service-Catalog: [{"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26:9696/networking"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/a8a6d900d9d344a694b955f51fa46aa9"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/volume/v3/a8a6d900d9d344a694b955f51fa46aa9"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2/a8a6d900d9d344a694b955f51fa46aa9"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/compute/v2.1"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.26/image"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.26:8080", "publicURL": "http://10.222.0.26:8080/v1/AUTH_a8a6d900d9d344a694b955f51fa46aa9"}]}] Oct 19 00:52:13.602180 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Tenant: tempest-TestMinimumBasicScenario-386785066 Oct 19 00:52:13.602180 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Tenant-Id: a8a6d900d9d344a694b955f51fa46aa9 Oct 19 00:52:13.602180 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-Tenant-Name: tempest-TestMinimumBasicScenario-386785066 Oct 19 00:52:13.602180 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User: tempest-TestMinimumBasicScenario-386785066-project Oct 19 00:52:13.602180 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User-Domain-Id: default Oct 19 00:52:13.602180 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User-Domain-Name: Default Oct 19 00:52:13.602180 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User-Id: 531efd120096452caf8c0b7a7f6b9ba1 Oct 19 00:52:13.602180 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: X-User-Name: tempest-TestMinimumBasicScenario-386785066-project Oct 19 00:52:13.602180 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: method: > Oct 19 00:52:13.602180 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: exception: Instance bfbf8641-41b8-47d8-af4d-ef17c6bd1935 could not be found. Oct 19 00:52:13.602180 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: action_args: {'id': 'bfbf8641-41b8-47d8-af4d-ef17c6bd1935'} Oct 19 00:52:13.605185 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Oct 19 00:52:13.605185 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 19 00:52:13.605185 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Oct 19 00:52:13.605185 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Oct 19 00:52:13.605185 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Oct 19 00:52:13.605185 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Oct 19 00:52:13.605185 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Oct 19 00:52:13.605185 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Oct 19 00:52:13.605185 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Oct 19 00:52:13.605185 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Oct 19 00:52:13.605185 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance bfbf8641-41b8-47d8-af4d-ef17c6bd1935 could not be found. Oct 19 00:52:13.605185 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:52:13.605185 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 19 00:52:13.605185 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:52:13.605185 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 19 00:52:13.605185 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 19 00:52:13.605185 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 19 00:52:13.605185 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Oct 19 00:52:13.605185 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Oct 19 00:52:13.605185 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Oct 19 00:52:13.608798 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Oct 19 00:52:13.608798 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Oct 19 00:52:13.608798 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi instance = self._get_server( Oct 19 00:52:13.608798 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Oct 19 00:52:13.608798 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Oct 19 00:52:13.608798 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Oct 19 00:52:13.608798 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Oct 19 00:52:13.608798 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance bfbf8641-41b8-47d8-af4d-ef17c6bd1935 could not be found. Oct 19 00:52:13.608798 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:52:13.608798 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Oct 19 00:52:13.608798 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:52:13.608798 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Oct 19 00:52:13.608798 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Oct 19 00:52:13.608798 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Oct 19 00:52:13.608798 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Oct 19 00:52:13.608798 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Oct 19 00:52:13.608798 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance bfbf8641-41b8-47d8-af4d-ef17c6bd1935 could not be found. Oct 19 00:52:13.608798 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: ERROR nova.api.openstack.wsgi Oct 19 00:52:13.610168 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-b5b33a43-a3ec-432e-8265-208c3eef3b89 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Returning 404 to user: Instance bfbf8641-41b8-47d8-af4d-ef17c6bd1935 could not be found. {{(pid=125641) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Oct 19 00:52:13.610168 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-b5b33a43-a3ec-432e-8265-208c3eef3b89 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] 10.222.0.26 "GET /compute/v2.1/servers/bfbf8641-41b8-47d8-af4d-ef17c6bd1935" status: 404 len: 111 microversion: 2.1 time: 0.086323 Oct 19 00:52:13.610168 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 937/1052] 10.222.0.26 () {58 vars in 1236 bytes} [Wed Oct 19 00:52:13 2022] GET /compute/v2.1/servers/bfbf8641-41b8-47d8-af4d-ef17c6bd1935 => generated 111 bytes in 89 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Oct 19 00:52:13.622274 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: DEBUG nova.api.openstack.wsgi [None req-c60ddb97-d86a-409a-ac45-435a98153917 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] Calling method '.version_select at 0x7f71c3926e50>' {{(pid=125641) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Oct 19 00:52:13.655993 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: INFO nova.api.openstack.requestlog [None req-c60ddb97-d86a-409a-ac45-435a98153917 tempest-TestMinimumBasicScenario-386785066 tempest-TestMinimumBasicScenario-386785066-project] 10.222.0.26 "DELETE /compute/v2.1/os-keypairs/tempest-TestMinimumBasicScenario-863658547" status: 202 len: 0 microversion: 2.1 time: 0.041969 Oct 19 00:52:13.657092 dv-cinder-master-3-iscsi devstack@n-api.service[125641]: [pid: 125641|app: 0|req: 938/1053] 10.222.0.26 () {58 vars in 1269 bytes} [Wed Oct 19 00:52:13 2022] DELETE /compute/v2.1/os-keypairs/tempest-TestMinimumBasicScenario-863658547 => generated 0 bytes in 45 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Oct 19 00:54:00.692039 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Oct 19 00:54:00.692840 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: Wed Oct 19 00:54:00 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers (ip 10.222.0.26) !!! Oct 19 00:54:07.441638 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Oct 19 00:54:07.442362 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: Wed Oct 19 00:54:07 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers (ip 10.222.0.26) !!! Oct 19 00:54:23.175152 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Oct 19 00:54:23.176073 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: Wed Oct 19 00:54:23 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers (ip 10.222.0.26) !!! Oct 19 00:57:20.704724 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Oct 19 00:57:20.705533 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: Wed Oct 19 00:57:20 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers (ip 10.222.0.26) !!! Oct 19 00:57:27.453180 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Oct 19 00:57:27.454000 dv-cinder-master-3-iscsi devstack@n-api.service[125640]: Wed Oct 19 00:57:27 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers (ip 10.222.0.26) !!!